Member of Technical Staff, Data & Evaluation

About Us
We are a small team of researchers and engineers focused on pushing the frontier of what agents are capable of. We believe the next leap in agent performance will come not from bigger general models, but from specialization and continual learning: agents that get better at their specific job the more they do it. We build the systems that make this possible, turning real-world usage into lasting improvement.

Summary
We are looking for researchers with a strong engineering background to lead synthetic data, environment generation and evaluation work on large language models, with a strong focus on agentic, real-world tasks. You will drive the research and implementation of building systems to generate and evaluate data with minimal human intervention. You will own data and evaluation quality end to end, from production usage and synthetic generation through to the evals that decide what ships. Your work will directly shape the agents our customers rely on in production.

Responsibilities

• Create and curate training data, including synthetic data and data drawn from production usage.

• Build training environments that reflect how agents are actually used in the real world, including long-horizon and multi-agent tasks.

• Design reward signals, graders, and verifiers for customer and domain-specific tasks.

• Build robust systems to detect and prevent reward hacking, false positives, and brittle verifiers.

• Design evaluations that reflect real production use, and use them to decide what works and what ships.

• Drive development of safe automated data and evaluation pipelines that run continuously and scale across customers and domain-specific tasks with minimal human intervention.

• Work closely with colleagues to understand how data and environments shape model behavior, and close the loop between eval signal and training decisions.

• Set research and engineering standards and mentor other engineers as the team grows.

Qualifications

• Deep, practical experience building training data, environments, or evaluations for language models, shown through shipped models, releases, or publications.

• Strong intuition for reward and grader design, and for how data choices affect model behavior, quality, and reliability.

• Experience with data or evaluation for reasoning or agentic tasks.

• Strong Python skills, and experience with large-scale data pipelines and distributed systems.

• Strong experimental design skills and a disciplined approach to evaluation.

• Able to deeply understand research papers and turn them into working, efficient code.

Nice to Have

• Synthetic data or evaluation experience at a frontier lab or on widely used open models.

• Publications at top ML conferences or well-known technical blog posts.

• Experience building environments or evaluations for long-horizon, multi-step tasks.

• Experience with continual learning or improving models from real-world feedback.