Internship | Remote | LLM Evaluation | Reports to CTO or Safety Lead
About Elloe
Elloe is the immune system for AI.
We don’t train models — we protect their outputs. We trace every hallucination, enforce every policy boundary, and create an audit trail for every critical LLM interaction.
Our modules (TruthChecker™, AutoRAG™, Autopsy™) are embedded in hospitals, banks, and regulatory sandboxes. Our job is to make sure these systems are safe before anything hits production.
This role will help us break, stress-test, and harden the models used by governments and enterprises alike.
About the Role
You’ll red team real-world LLM deployments, design eval harnesses, and help scale Elloe’s output-level safety layer. This isn’t just prompt tuning — it’s forensic risk mapping.
You’ll work directly with product and safety leads to uncover failure patterns and codify guardrails for GenAI systems under real-world scrutiny.
What You’ll Own
1. Red Teaming & Risk Testing
- Create prompts to trigger hallucinations, policy violations, or failure scenarios
- Stress test Elloe-protected deployments using open and proprietary models
- Document behavioral exploits across use cases (healthcare, compliance, gov)
2. Evaluation Design
- Build truthsets and scoring rubrics tied to factuality, policy, or ethical standards
- Benchmark Elloe’s modules across model types (Claude, GPT-4, Gemini, open models)
- Collaborate with product to refine and expand our eval harnesses
3. Safety Intelligence
- Identify blind spots in current detection logic
- Recommend scoring methods or red flag thresholds for deployment
- Support internal model comparison reports or customer safety audits
Who You Are
- ML/AI researcher or engineer (undergrad, grad, or early career)
- Experience working with LLMs, eval sets, and prompt design
- Strong attention to detail, grounded in safety and adversarial thinking
- Bonus: exposure to safety benchmarks like TruthfulQA, MMLU, or red teaming tools
Why This Matters
This is real-world alignment, not research theater.
You’ll be helping define how AI gets deployed responsibly — with traceability, transparency, and real-time protection.
You’ll leave this role with:
- Exposure to high-stakes LLM safety deployments
- Published frameworks or scoring methods used by enterprises
- Mentorship from technical founders operating at the bleeding edge of AI safety
Logistics & Application
- Start Date: Rolling
- Duration: 12–16 weeks
- Compensation: Research stipend
- Location: Remote-first; flexible for global candidates
- To Apply: Share a jailbreak or eval idea you’d love to run against GPT-4.
