Remote AI Quality Analyst

About the role

We are looking for a Remote AI Quality Analyst to evaluate a new personalization feature for an AI model. The role involves using your own experiences as context to test how well the model integrates data from past conversations, email, search and activity to deliver relevant, helpful responses.

Key responsibilities

• Evaluate the quality of personalized AI responses using personal experiences as context.
• Design creative, multi‑turn prompts that test the model’s ability to integrate personal data sources accurately.
• Assess model outputs for grounding, integration, helpfulness, and identify incorrect personalization or forced connections.
• Review side‑by‑side comparisons of AI responses to detect subtle differences in relevance and accuracy.
• Provide detailed feedback on model behavior to improve personalization performance.

Required profile

• High proficiency in English reading and writing.
• Experience in prompt engineering or designing test cases for AI models.
• Strong analytical ability to evaluate nuanced and ambiguous AI responses.
• Familiarity with personalization concepts and AI evaluation metrics.
• Attention to detail for spotting subtle discrepancies in model outputs.
• Willingness to use a personal Google account and enable personal data sources for assessment.
• Full‑time availability in your local time zone to support a 24‑hour global operations team.

Required skills

• Prompt engineering

What we offer

• Competitive compensation based on experience.
• Full‑time remote work with flexible hours.
• Opportunity to contribute to a global leader in technology and AI.