Specialist – Data QA

Job Summary

Join Almosafer as a Data QA Specialist to ensure the quality, reliability, and performance of data, AI, and machine learning solutions. You will design and execute data quality checks, validate AI/ML models, and support testing for generative AI and LLM applications.

Responsibilities

• Design and execute data quality checks; validate data for accuracy, completeness, and consistency; perform source-to-target validation and reconciliation.
• Use SQL and Python to investigate data anomalies and develop automated testing frameworks.
• Support regression testing for production data and ML solutions.
• Evaluate AI/ML models against business requirements and analyze outputs for inconsistencies.
• Support testing for model performance, data drift, bias, and reliability alongside Data Scientists and ML Engineers.
• Test GenAI, LLM, and RAG applications for accuracy and factuality; identify hallucinations and bias.
• Perform prompt and scenario-based testing and develop test data for continuous quality monitoring.
• Partner with Data Scientists, Data, ML, and Software Engineers, and Product teams to define quality standards, test strategies, and acceptance criteria; perform root-cause analysis on defects.

Requirements

• Bachelor’s degree in Computer Science, Data Science, Engineering, or a related field.
• Experience in Data QA, Data Science, Data Engineering, Machine Learning, or AI/ML Testing.
• Strong Python and SQL programming skills.
• Experience with automated testing, preferably PyTest or similar frameworks.
• Solid understanding of data quality concepts and machine learning fundamentals.
• Strong knowledge of prompt engineering and experience designing and evaluating prompts for AI/LLM applications.
• Working knowledge of MCP (Model Context Protocol) servers and how they integrate with AI/LLM applications.
• Experience working with skills files / AI agent configuration and instruction files and understanding how they are used to guide AI workflows.
• Hands-on experience with Claude Code or similar AI-assisted development tools.
• Strong analytical, problem-solving, and communication skills.

Preferred Qualifications

• Hands-on experience with Generative AI, LLM applications, RAG architecture, and AI evaluation frameworks such as Ragas, TruLens, or DeepEval.
• Familiarity with LLM platforms such as OpenAI, Anthropic, and Llama, as well as concepts such as data/model drift and bias monitoring.
• Experience with cloud platforms such as AWS, Azure, or GCP and CI/CD processes.

What the company offers

• A purpose-led workplace where your ideas can create real impact.
• Ongoing learning and development to help you grow with intention.
• Recognition that celebrates your wins, both big and small.
• A community built on trust, curiosity, and shared momentum.

We refresh listings regularly, but some roles close early on the source platform.