About the Role
Join a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. You will own the design and implementation of domain-specific evaluations that help research teams and customers understand how agents perform on realistic workflows.
What You'll Do
- Design, implement, and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks.
- Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria.
- Build reliable infrastructure to run models and agents against benchmark tasks at scale.
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
- Validate whether benchmark results align with real-world evaluations and user needs.
- Write clear technical documentation and reports explaining benchmark methods and results.
What We're Looking For
- Experience: Two to four years in software engineering, machine learning engineering, or research engineering, including at least two years building AI benchmarks, evaluation infrastructure, or agent environments.
- Education: Technical education or strong fundamentals in a technical field.
- Technical skills: Proficiency with Python, Docker, and Linux for research or production infrastructure.
- AI expertise: Hands-on experience designing and operating evaluations or environments for AI agents or large language models.
- Infrastructure and analysis: Experience building reliable model or agent execution infrastructure, developing evaluation metrics or statistical analyses, and validating benchmark performance.
- Domain collaboration: Ability to work with subject-matter experts and understand workflows across technical or business domains.
- Soft skills: Strong technical writing, attention to detail, independent problem-solving, and comfort in an early-stage environment.
- Valued experience: Published research or technical writing on AI benchmarks, evaluation methods, or model failure modes. Reinforcement learning pipelines, data generation, and agent evaluation experience are also relevant.
Compensation & Benefits
- Base salary: $100,000 to $170,000 USD annually.
- Visa sponsorship: Available.
Location
On-site in Singapore, Singapore.
About Clera
Clera is an AI talent agent that matches candidates directly to startups based on their goals, skills, and preferences, and provides personalized introductions. Candidates share their experience, preferences, and dealbreakers through chat channels including iMessage and WhatsApp, and the agent makes warm introductions directly to founders and hiring managers at more than 600 venture-backed startups. The service is free for candidates, and companies pay Clera when they hire an introduced candidate.
Interested in this role?
Apply now to join Clera.