MLPerf Software Engineer at MLCommons Association
About MLCommons Association
MLCommons is a new kind of organization for the AI era. We are a non-profit that builds the standards that define AI through open, state-of-the art, industry-standard benchmarks to measure quality, performance, and risk. We aren’t like standards organizations that slowly negotiate over specifications - we bring the community together to work on cutting edge problems and build the future with code.
We’re a global consortium supported by 125+ members and affiliates, including startups, leading companies, and academics. We are a fully remote organization that supports a flexible working environment; this requires mature communication, accountability, and transparent collaboration by all team members.
About MLPerf
The MLPerf family of benchmarks is the industry standard for measuring performance and efficiency of AI on everything from smartphones and laptops to the world’s largest supercomputers. Our open, fair, and trusted benchmarks help bring together the broader community, including researchers, engineers, purchasers, and vendors to enable more capable and efficient AI for the entire world.
The MLPerf team includes product managers, program managers, and engineers who work collaboratively with member organizations. Together, they build, operate, and sustain the MLPerf benchmarks, AI workload evaluation harnesses, and infrastructure that serve our members, their stakeholders, and the entire AI ecosystem.
You can read about MLPerf here:
Scope of MLPerf Software Engineer
As an MLPerf Software Engineer, you will work in the open-source space, designing and building the systems and infrastructure that power the MLPerf benchmarks — the automation behind the benchmarks and the critical processes around submission, review, publication, and reproducibility, as well as the visualization of benchmarking results.
This is a role for an engineer who thinks in systems: someone who can own architecture and technical direction, weigh trade-offs, and design infrastructure that scales — not just implement against a spec. You will work alongside the MLPerf team and experts in performance optimization, benchmarking, and ML at our member organizations to shape the evolution and success of the MLPerf benchmarking suite.
Responsibilities
- Architect and design benchmarking systems, infrastructure, and tooling — owning technical decisions from concept through implementation.
- Design robust, scalable automation for submission, validation, review, publication, and reproducibility of benchmark results.
- Learn and operate existing MLPerf pipelines; identify structural improvements and re-architect where needed rather than patch around them.
- Partner with MLPerf engineering and program management to translate roadmap goals into technical designs and deliver them.
- Collaborate with ML engineers, DevOps teams, and hardware vendors to design and optimize end-to-end benchmarking pipelines.
- Debug and resolve complex issues across automation, submissions, and benchmark harnesses.
- Produce high-quality design docs and documentation to drive collaboration and technical alignment.
- Present technical designs, progress, and outcomes to the MLPerf working groups.
Required Qualifications
- Demonstrated experience designing systems and infrastructure — architecture decisions, trade-off analysis, and building for scale and reliability.
- 4+ years of post-academic software engineering (or equivalent impact via open source / industry).
- Proficiency in Python and Bash; strong automation mindset.
- Hands-on experience with GitHub Actions (or equivalent CI/CD), artifact/version management.
- Working knowledge of containers (Docker) and running workloads reproducibly.
- Familiarity with ML concepts (inference/training basics) and performance benchmarking or optimization.
Nice-to-have Qualifications
- Experience designing distributed systems, data pipelines, or large-scale automation infrastructure.
- Infrastructure-as-code (e.g., Terraform) and cloud platforms.
- Experience with MLPerf (reference or vendor implementations), LoadGen, and logging schemas.
- ML frameworks: PyTorch, TensorFlow, ONNX Runtime.
- Understanding of hardware for ML (CPUs/GPUs/TPUs/accelerators) and performance profiling.
- Data validation, schema management, and results visualization experience.
- Prior open-source contributions in benchmarking tools and communities.
Why Join Us
- Own the design of open, industry-standard tooling and infrastructure that advances trustworthy, efficient AI.
- Collaborate with leading hardware vendors, ML engineers, and researchers worldwide.
- Shape the architecture and best practices for automation, reproducibility, and software quality at scale.
- Fully remote with high-ownership work and meaningful community impact.
Compensation & Timeline
- Type: Contractor (non-W2)
- Location : Remote (North American time zone preferred)
- Commitment: ~40 hrs/week
- Duration: 12 months, with possible extensions
- Start date: Immediately
- Eligibility: We currently do not provide Visa Sponsorship
- Compensation: commensurate with experience
Application Process
Send the following to [email protected] with subject: "Applying for MLPerf Software Engineer Role — [Your Name]"
- Resume/CV
- Relevant links (GitHub, LinkedIn, personal site)
- Answer to the question: "Think of a system or automation you designed. If you were to redesign it today, what's one thing you would do differently and why?"
- Where you saw this role posted or how you learned about the opening (e.g., MLCommons newsletter, LinkedIn, referral, etc.)
Thanks. We look forward to hearing from you!
