Automation Test Engineer

Position Title

Automation Test Engineer – AI/ML Output Validation (1 Position)

Salary Grade

Alpha numeric code (if any)

Job Position Summary

We are seeking a highly skilled Automation Test Engineer to design build and maintain automated testing frameworks that validate AI/ML-generated outputs for source-code internationalization (i18n) issue detection. The candidate will be responsible for ensuring accuracy consistency and reliability of model predictions and generated remediation suggestions across multiple programming languages. This role combines strong automation expertise with a deep understanding of software testing methodologies source-code behavior and AI/ML validation practices.

Key responsibilities

1. Develop automated test scripts to validate AI/ML outputs related to i18n issue detection and recommended code fixes.
2. Build scalable test frameworks to evaluate model precision recall F1 score and false-positive/false-negative behaviors.
3. Create automated pipelines for regression testing of updated AI/ML models.
4. Design and execute test plans test cases and acceptance criteria for multi-language codebases (e.g. Ruby Python TypeScript Kotlin/Swift).
5. Validate correctness safety and quality of AI-generated code remediation suggestions.
6. Compare AI predictions against expected outputs using rule-based static-analysis and human-verified datasets.
7. Integrate automated tests into CI/CD pipelines to ensure continuous model evaluation.
8. Log defects inconsistencies and model deviations using standardized workflows.
9. Collaborate closely with AI/ML Engineers Static Analysis Engineers and Software Developers to improve model accuracy and test coverage.
10. Maintain documentation for test frameworks testing tools and processes.

Desired Qualification / certifications

1. Bachelor’s or master’s degree in computer science Software Engineering Information Systems or related fields.
2. Certifications in Software Testing Automation Tools or ISTQB preferred.
3. Training in AI/ML fundamentals is a plus.

Preferred relevant work experience

1. 3–4 years of experience in automation testing or QA engineering.
2. Hands-on experience testing systems that produce AI/ML outputs text generation or rule-based predictions.
3. Experience working with or testing static code analyzers linters or compilers is strongly preferred.
4. Previous work validating code correctness or scanning tools (e.g. ESLint Pylint) is advantageous.

Skills and knowledge

Technical Skills

1. Strong programming skills in Python or similar languages for automation scripting.
2. Experience with test automation frameworks (PyTest or custom frameworks).
3. Ability to build automated evaluation harnesses comparing expected vs. actual AI outputs.
4. Familiarity with source-code structures (ASTs syntax patterns code smells linting rules).
5. Understanding i18n concepts localization patterns and common i18n coding issues.
6. Proficiency with GitHub/GitLab CI/CD for automated test execution workflows.
7. Experience working with APIs JSON data validation and structured output comparisons.
8. Strong debugging and problem-solving skills in multi-language codebases.

General Skills

1. Excellent attention to detail and commitment to accuracy.
2. Strong analytical thinking and ability to identify subtle model or logic inconsistencies.
3. Effective communication skills for collaborating across AI engineering and QA teams.
4. Ability to work independently and in fast-paced iterative development cycles.

Position reports to

Program Manager

Position supervises

Nil

Travel needed

Nil

Location

Fidel Softech Pune office Or client offices in India or abroad

Employment details

Full time and on the rolls of the Fidel Softech