This opportunity is ideal for senior software engineers with strong Python expertise who enjoy writing high-quality code, reviewing technical solutions, and working on AI-related projects.
What You'll Do:
- Design and develop coding benchmarks used to evaluate frontier AI models.
- Analyze AI-generated code for correctness, reliability, efficiency, and edge cases.Build and maintain scalable data pipelines that support AI evaluation workflows.
- Create structured programming scenarios to test reasoning, debugging, and code quality.
- Work with large codebases and multi-language software environments.
- Collaborate with teams focused on improving how AI models understand, generate, and evaluate software.
- Write clean, maintainable, and well-tested Python code following software engineering best practices.
Requirements:
- 4+ years of professional software engineering experience (required).
- Expert-level proficiency in Python.
- Experience working at a high-growth technology company or top-tier software organization.
- Proficiency in at least one additional programming language such as JavaScript, Go, C++, or similar.
- Experience with CI/CD pipelines and automated testing frameworks such as pytest, Mocha, or JUnit.
- Strong understanding of software engineering best practices, debugging, and code quality.
- Excellent analytical and problem-solving skills.
Nice to Haves:
- Experience with AI/ML evaluation, model benchmarking, or Generative AI.
- Background in security engineering.
- Significant contributions to open-source software projects.
- Experience working with large-scale distributed systems or enterprise software platforms.
- Fully remote contract opportunity.
- Compensation ranges from $80–$100 USD per hour.
- Expected workload is 10–39 hours per week, depending on project needs.
- Weekly payments for approved work completed during the previous week.
- Work volume may fluctuate throughout the duration of the engagement.
Hiring Process:
- Submit your proposal.
- Complete a short qualification form provided by the client.
- Qualified candidates will receive an Upwork contract offer and onboarding instructions.
- After accepting the contract, candidates will gain access to the client's evaluation.
- Candidates who successfully pass the evaluation will be invited to a technical interview with the client.
