Evaluation QA Lead (Agentic and RAG Evaluation)

Vettedlance

New Remote

Contract Remote Senior 60 - 90 USD/hour 🇬🇧 English

Job description

About the role

Vettedlance is hiring an experienced QA lead to run quality for AI evaluation projects, covering rubric design, reviewer calibration and agreement tracking for agentic and RAG systems. This is a remote, part-time independent contractor position.

Key responsibilities

Own QA for agentic-workflow and RAG/conversational evaluation projects.
Write and refine rubrics and gold sets.
Calibrate reviewers and track inter-annotator agreement (kappa, alpha).
Audit samples and report quality trends.

Required profile

4+ years owning QA on annotation, evaluation or software testing.
Hands-on experience evaluating LLM, agent or RAG systems.
Fluent written English.

Required skills

Comfortable with agreement metrics and spreadsheets.

What we offer

Remote work, 10-40 hours per week.
Hourly rate of $60-$90 USD, paid weekly; rate set after a short paid assessment.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec Vettedlance.

Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Le contrat proposé est un Contract.

Why are you reporting this job?

Thank you for your report. We will review this job.

Explore further

Salaries, guides and searches in Pakistan.

Salaries by job title

Business Development Executive 101
Sales Executive 89
Graphic Designer 45
Social Media Manager 43
Business Development Manager 42
Customer Service Representative 40
Sales Development Representative 36
Software Engineer - E-Commerce (Remote) 30

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 12 hours ago

Expires 1 month from now

5 views · 0 interested

Share Log in to earn credits by sharing

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...