Rubric Evaluation Specialist - Writing

Job Description

About the job

\n

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

\n

Position: Generalist Writer
Type: Contract
Compensation: $25–$45/hour
Location: Remote

\n

Role Responsibilities

\n

\n
• Evaluate AI-generated written conversations using predefined rubrics to enhance performance.
\n
• Draft and refine evaluation rubrics for generalist writing tasks.
\n
• Apply structured scoring criteria to assess clarity, coherence, accuracy, and style.
\n
• Review edge cases and ambiguous outputs to ensure consistent grading standards.
\n
• Work independently and asynchronously to meet deadlines and improve AI model performance .
\n

\n

Qualifications

\n

Must-Have

\n

\n
• Experience with rubric-based grading , editing, or content evaluation at Mercor .
\n
• Strong generalist writing skills across multiple domains (e.g., business, academic, creative).
\n
• Excellent attention to detail and consistency in applying guidelines.
\n
• Ability to interpret nuanced criteria and make sound judgment calls.
\n

\n

Preferred

\n

\n
• Background in English , journalism, communications, or a related field.
\n

\n

Compensation & Legal

\n

\n
• Opportunities for expanded reviewer/lead responsibilities.
\n

\n

Application Process (Takes 20–30 mins to complete)

\n

\n
• Submit a resume or writing sample to get started.
\n
• Selected applicants will be contacted within a few days regarding next steps.
\n

\n

Resources & Support

\n

\n
• For details about the interview process and platform information, please check:
\n
• For any help or support, reach out to:
\n

\n

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.