AI Quality Analyst – Remote

About the role

We are seeking an AI Quality Analyst to evaluate a new personalization feature for an AI model. Working remotely, you will use your own experiences as context to test how well the model integrates data from past conversations, email, search, and activity into relevant responses.

Key responsibilities

• Evaluate the quality of personalized AI responses using personal experience as context.
• Design creative, multi‑turn prompts that challenge the model’s ability to integrate personal data sources.
• Assess model outputs for grounding, integration, helpfulness, and identify incorrect personalization.
• Review side‑by‑side AI response comparisons to detect subtle relevance and accuracy differences.
• Provide detailed feedback to improve the model’s personalization performance.

Required profile

• Strong English reading and writing proficiency.
• Experience in prompt engineering or designing test cases for AI models.
• Analytical ability to evaluate nuanced and ambiguous AI responses.
• Familiarity with personalization concepts and AI evaluation metrics.
• Attention to detail for spotting subtle output discrepancies.
• Willingness to use a personal Google account and enable personal data sources for testing.
• Full‑time availability in your local time zone to support a 24‑hour global operations team.

Required skills

• Prompt engineering
• AI model evaluation

What we offer

• Competitive compensation based on experience.
• Opportunity to work with a global leader in technology, information, and internet services.
• Remote work from anywhere.