AIML Automation Testing Engineer (RARR Job 6467)

Primary Skillsets
     Automation Testing with 8-12 years of experience in designing developing and executing automation frameworks for AIML driven applications. The ideal candidate brings handson experience with GenAI Testing LangSmith, PyTest Agentic frameworks and strong AIML evaluation expertise.

Detailed JD

Role Overview
We are seeking a Lead Software Engineer  Automation Testing with 8-12 years of experience in designing developing and executing automation frameworks for AIMLdriven applications. The ideal candidate brings handson experience with GenAI Testing, LangSmith, PyTest Agentic frameworks and strong AIML evaluation expertise.

Key Responsibilities
AI  GenAI Testing
Design and execute VV strategies for classical ML and GenAI agentic systems.
Define test plans tests cases and execute them.
Evaluate AI models for accuracy performance safety robustness hallucinations bias and correctness.
For GenAI rely on LangSmith for LLM observability tracing monitoring and evaluation for offline and online evaluation.
For classical ML UC ensure VV tests are done during the whole life cycle of the model ML monitoring
Test Automation
Build scalable automation frameworks using PyTest integrated with CICD pipelines
Develop automated test suites for APIs microservices and AI pipelines
Perform sanity integration functional regression and UAT testing across QA UAT and Prod environments
Implement automation strategies Keyword driven, Datadriven BDD Hybrid frameworks
Quality Assurance  Reporting
Analyze test results and provide insights to engineering teams
Log and track defects using standard bug reporting tools
Maintain high quality documentation for all test processes and reports