AI/ML Quality Assurance Engineer

AI/ML Quality Assurance Engineer

Industry: Healthcare

Location: Virginia / U.S.

Employment Type: Contract

Role Summary

seeking an experienced AI/ML quality assurance engineer to support the testing and validation of machine learning and AI solutions within a healthcare environment. The role will work closely with data scientists, ML engineers, data engineers, and application teams to help ensure AI solutions are accurate, reliable, scalable, and production-ready.

Key Responsibilities

• Develop and execute test strategies for AI/ML models and AI-enabled applications.
• Validate model inputs, outputs, data transformations, and downstream integrations.
• Perform functional, integration, regression, and end-to-end testing of AI/ML solutions.
• Validate large healthcare datasets used for model development and testing.
• Assess data completeness, consistency, accuracy, and suitability for AI/ML workloads.
• Compare model behavior across different datasets and test scenarios.
• Identify model-quality issues, data anomalies, unexpected outputs, and regression risks.
• Partner with Data Science and ML Engineering teams throughout model development and deployment.
• Develop repeatable testing and validation processes for ML pipelines.
• Support testing of both structured and unstructured healthcare data.
• Validate APIs and data pipelines supporting AI/ML applications.
• Track defects, document findings, and work with engineering teams through resolution.
• Support model performance and production-readiness validation.
• Contribute to automation of AI/ML testing and data-validation workflows.

Required Skills

• Strong experience in quality engineering/software testing.
• Experience testing AI, machine learning, or data-intensive applications.
• Strong understanding of data validation and data-quality testing.
• Experience working with large datasets and complex data pipelines.
• Strong SQL skills.
• Experience with Python for testing, automation, or data validation.
• Experience with API testing and backend validation.
• Understanding of machine learning development and model lifecycle concepts.
• Ability to work closely with data scientists, ML engineers, and data engineers.
• Strong analytical and problem-solving skills.
• Experience working within Agile development environments.

Healthcare Experience

• Experience working with healthcare data is strongly preferred.
• Familiarity with healthcare datasets such as claims, member, provider, clinical, laboratory, or population-health data is beneficial.
• Understanding of healthcare data privacy, security, and data-quality considerations is preferred.

Preferred Skills

• Experience validating AI/ML models using real-world datasets.
• Exposure to model performance, bias, accuracy, drift, and regression testing.
• Experience with automated data quality or ML-validation frameworks.
• Exposure to cloud-based AI/ML platforms.
• Experience with CI/CD and automated testing pipelines.
• Familiarity with Generative AI or LLM testing is a plus.

Pay: $60.00 - $70.00 per hour

Work Location: In person