Principal Test Engineer – AI Core Services

Job Title: Principal Test Engineer – AI Core Services

Work Location: Cambridge, UK (Hybrid, 3 Days Onsite)

Role Overview:

We are looking for a Senior Principal Test Engineer to ensure the quality, performance, and security of our Core AI Services. This role focuses on validating distributed, cloud-native services and public APIs, with a strong emphasis on AI-specific testing, automation, and compliance.

You will work within a Scrum team, collaborating with developers, architects, and data scientists to design validation strategies, automated frameworks, and AI evaluation tools. This position combines deep technical expertise with a builder mindset, enabling rapid prototyping and continuous improvement.

Key Responsibilities:

• Lead functional, performance, and security testing of Azure-based cloud-native services and APIs.
• Design & implement automated test suites for APIs, microservices, and AI pipelines.
• Automate AI system evaluations to ensure accuracy, consistency, and safety.
• Define and validate service-level quality metrics and observability hooks.
• Ensure compliance with AI regulatory frameworks (EU AI Act, NIST AI RMF).
• Contribute to threat modelling and security validation.
• Provide early lifecycle feedback to improve design and reduce defects.
• Mentor junior engineers and promote a culture of innovation and ethical AI.
• LLM evaluation, output scoring, hallucination detection, and validation frameworks.
• Knowledge of prompt engineering, RAG, and model orchestration.
• AI performance metric definition and testing.
• Awareness of AI safety, bias detection, and explainability techniques.
• Strong belief in ethical AI practices and compliance.

Core Skills & Qualifications:

• 8-12+ years in software testing/QA for cloud-native apps using Microsoft & .NET.
• Proven track record in test automation frameworks and CI/CD with Azure DevOps.
• Expertise in API testing, performance profiling, and security validation (OWASP Top 10).
• Strong analytical and problem-solving abilities.
• Agile/Scrum experience in global R&D environments.
• Demonstrated mentorship and leadership skills.