About this role
Phone numbers and emails in this ad are masked until you log in.
auto_translated_note
Role Title: AI Rubric WriterRole Type: ContractorLocation: RemoteWe are seeking an experienced AI Rubric Writer to develop high-quality evaluation rubrics and scoring frameworks for AI and Large Language Model (LLM) assessments. The ideal candidate will create clear, objective evaluation criteria to assess AI-generated responses and help improve model performance across a variety of subject areas.Responsibilities Design and develop evaluation rubrics and scoring guidelines for AI-generated content. Create objective, consistent, and well-structured assessment criteria.
Evaluate AI responses for accuracy, reasoning, completeness, and quality. Provide detailed feedback to improve AI model performance. Collaborate with cross-functional teams to refine evaluation standards.
Ensure rubrics are clear, unbiased, and easy to apply. Required
Qualifications
Bachelor's degree or higher in a relevant field. Strong analytical, critical thinking, and written communication skills. Experience creating evaluation criteria, scoring frameworks, assessment guidelines, or rubrics.
Subject matter expertise in Software Engineering, Finance, Law, Consulting, or other quantitative/STEM disciplines. Preferred
Qualifications
Experience as an AI Evaluator, Prompt Evaluator, or similar AI quality assessment role. Experience evaluating Large Language Models (LLMs) or Generative AI systems. Familiarity with rubric development, AI quality assurance, or data annotation.
Experience reviewing and assessing technical or domain-specific content. Originally posted on Himalayas
Community Q&A
Anyone worked here? Ask before you apply.
No threads yet for this job or company.