Sr. AI/ML QA Automation Engineer

Sr. AI/ML QA Automation Engineer

Cupertino, California, United States

The Apple Photos application is a comprehensive photo and video management solution that seamlessly integrates across the entire Apple ecosystem, enabling users to capture, organize, edit, and share their visual memories with unprecedented ease and intelligence. Working on the Photos team means contributing to one of Apple's most personal and widely-used applications, combining cutting-edge AI, elegant design, and robust engineering to help billions of users around the world preserve and relive their most important life moments. This is a high-impact role where you'll work at the intersection of AI modeling, agentic workflows, information retrieval, software engineering, evaluation and metrics, and help us push the boundaries of how AI can transform Apple's products.

Description

This role blends traditional software QA skills with advanced evaluation methodologies for modern AI models, including LLMs, multimodal systems, and ML-driven product features. As a member of the team, you will work closely with experienced engineers and machine learning experts to qualify and refine features powered by vision, language, and cross-modal intelligence. You will be responsible for designing rigorous evaluation strategies for both objective and subjective ML behaviors, creating reliable automated testing pipelines, and developing LLM-driven evaluators that complement human judgement. The ideal candidate is self-directed, creative, and comfortable with ambiguity, with strong technical and interpersonal skills. They have hands-on experience testing ML models directly, defining qualitative scoring rubrics, building reproducible evaluation frameworks, and ensuring that AI behavior is safe, consistent, and aligned with product and on-device constraints.

Responsibilities

Design and implement test strategies for qualifying subjective and ML-driven features

Perform functional testing of existing/new features on various platforms across concurrent release vectors

Design and execute evaluation plans for multimodal ML models (vision, language, and cross-modal systems)

Develop and maintain robust testing frameworks for machine learning models

Automatic large scale data generation & evaluation

Collaborate with QA teams to integrate models into existing test frameworks

Consider on-device ML performance constraints (latency, memory, energy) when designing tests and evaluation strategies

Minimum Qualifications

BS/MS or equivalent experience in Computer Science or related field

3+ years of experience working in Software Quality Assurance

Strong software engineering skills, including system design, development, testing, debugging, release and maintenance

Expertise with hands on experience in automated software testing, data validation, and ethical AI practices

Familiarity with LLM usage to improve efficiency of their daily work

Ability to evaluate ML models directly (vision or multimodal) and diagnose model failures using quantitative and qualitative methods

Expertise in Python

Proficiency with iOS, macOS, watchOS, tvOS or similar operating systems

Preferred Qualifications

Experience testing AI models for accuracy, robustness, fairness, and performance

Experience developing LLM based automated evaluation frameworks

Expertise in Swift and/or Obj-C

Strong programming skills in Python and experience with ML/NLP libraries

3+ years of proven ability in machine learning, including hands-on work with LLMs

Understanding of prompt engineering, and retrieval-augmented generation (RAG)

Knowledge of statistics based evaluation approaches, ML training pipelines and accuracy improvements of ML systems

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $147,400 and $272,100, and your base pay will depend on your skills, qualifications, experience, and location. Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation.

Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other legally protected characteristics.