Job Description: Software Engineer – AI Training & Model Evaluation

Role Type: Contractor
Location: Global (Fully Remote)
Time Commitment: 15+ Hours per Week
Schedule: Flexible

About the Role

We are seeking experienced Software Engineers with prior professional AI training or model evaluation experience to contribute to a project focused on improving advanced AI systems.

In this role, you will apply your software engineering expertise to create challenging technical tasks, evaluate AI-generated code, identify technical failures, and establish objective standards for correct solutions. This opportunity is intended for professionals with substantial prior experience in AI training, benchmarking, or model evaluation.

Key Responsibilities

• Create realistic software engineering tasks that challenge advanced AI models.

• Develop detailed task instructions, reproducible environments, reference solutions, tests, and evaluation criteria.

• Evaluate AI-generated code and technical deliverables for correctness, robustness, and completeness.

• Identify issues related to implementation, debugging, dependencies, performance, testing, and reproducibility.

• Assess technical solutions against established software engineering standards.

• Provide clear feedback to support improvements in AI model performance and reliability.

Required Skills and Qualifications

• Prior paid professional experience in training, evaluating, benchmarking, or improving AI systems.

• Strong practical proficiency in Python.

• Professional experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.

• Strong understanding of software design, debugging, testing, version control, and dependency management.

• Ability to develop, review, and validate complex software engineering solutions.

Public Proof of Technical Expertise

Strong applicants should provide publicly verifiable evidence of their technical skills, such as:

• Substantial GitHub or GitLab repositories.

• Meaningful open-source contributions.

• Maintainer or core-contributor responsibilities.

• Codeforces, ICPC, Olympiad, or other programming competition results.

• Public developer tools, libraries, systems, or technical projects.

• Technical publications, benchmarks, or engineering write-ups.

Preferred Qualifications

• Experience creating AI training tasks, reference solutions, tests, graders, or benchmark problems.

• Experience reviewing technical submissions from other AI trainers.

• Codeforces rating of approximately 1900 or higher, or strong results in another programming competition.

• Experience with Linux, Docker, CI/CD, command-line tools, or sandboxed environments.

• Prior experience with long-horizon or agentic coding evaluations.

• Availability to begin immediately.

Application Process

1. Submit your application, including details of your AI training experience and publicly verifiable technical contributions.

2. Complete an AI interview involving technical and integrity checks.

3. Undergo a Hiring Manager review.

4. If selected, complete project onboarding and begin your first assigned task.