The AI Assurance Quality Engineer plays a key role in validating the performance, accuracy, and governance of artificial intelligence systems. You will focus on testing AI-generated content for correctness, relevance, and potential risks such as hallucinations or bias, ensuring outputs meet functional and ethical standards.
Key Responsibilities
- Run test cases to evaluate AI outputs, including accuracy, consistency, and adherence to expected behavior
- Support validation of Retrieval-Augmented Generation (RAG) pipelines and prompt logic
- Test APIs and structured data formats like JSON to confirm correct data handling and system integration
- Help build and maintain automated test scripts for regression and functional testing of AI features
- Log defects, track quality risks, and contribute to audit-ready documentation for AI governance
- Engage in agile workflows, including test planning and cross-team coordination
- Verify accessibility and user experience in AI-powered interfaces
Qualifications and Skills
You should have a solid foundation in software testing principles, with hands-on experience in API and data validation. Proficiency in Python or JavaScript is required for developing test automation. A basic grasp of large language models and prompt engineering is essential, along with strong attention to detail when analyzing AI behavior.
- Degree in Computer Science, Software Engineering, Data Science, or equivalent experience
- Understanding of Git for version control
- Strong written communication skills for documenting issues and quality observations
- Azure AI certification is required
- Demonstrated interest in AI, data quality, or test automation
Preferred Background
- Experience with test automation tools such as Playwright or Cypress
- Familiarity with cloud platforms like AWS, Azure, or GCP
- Knowledge of CI/CD pipelines and DevOps practices
- Exposure to Responsible AI concepts including fairness, explainability, and bias detection
- Testing experience with machine learning or data-driven applications
