Aleroy is building a unified platform for high-performance AI inference, making it easier for companies to deploy and run AI models at scale.
Through a single API, Aleroy provides access to ready-to-use and custom AI models across cloud infrastructure, dedicated capacity, and customers' own hardware. Our platform optimizes performance, cost, and deployment flexibility while giving enterprises control over where and how their AI workloads run.
We work in small teams, collaborate directly, and focus on solving complex engineering challenges to build reliable, scalable AI infrastructure.
You build the platform our data-centre partners run. You bring it up on their clusters, keep each customer isolated, route and meter every request, and run the sandboxes where customers' AI agents execute code. This is systems work in Go on Kubernetes and Linux; it needs no ML research.
We're looking for a Senior / Software Engineer, Inference Platform & Agent Sandboxes. You work with our Tech Lead and CTO, and next to the engineer who owns model performance.
Running on partner clusters
Bring-up on a new partner's hardware; failure detection and recovery
Deploying and scaling model servers on Kubernetes: autoscaling, rollouts, GPU placement
Isolation between customers: data, cache and limits
Request path
The OpenAI-compatible API and token streaming, with cancellation and client disconnects
Cache-aware routing across replicas, models and backends; admission control and backpressure when request cost varies widely
Usage metering per customer and per partner, counted exactly once, because partners are paid on it
Agent sandboxes
Isolated environments where customers' AI agents run code: fast start, snapshot and restore, resource limits, network policy
Running them on partners' CPU and GPU capacity, with the same isolation and metering as inference
Training jobs and customer data
Post-training jobs on partner GPUs: multi-node scheduling, checkpointing and recovery
Customer data and model weights kept in the customer's own storage, with customer-managed encryption keys
3+ years (mid) or 5+ years (senior) building and operating distributed systems in production
Strong production experience in Go; TypeScript (Node.js) or Python is a plus
Kubernetes in production: scheduling, autoscaling, rollouts; writing operators or controllers is a plus
Linux internals: containers, cgroups, namespaces, networking
Streaming APIs and request routing where request cost varies widely
You have owned a production service: on-call, incidents, rollbacks
For senior: you have led the design of a system end to end and explained its trade-offs to other teams
Professional working English
Real ownership of the backend that powers Aleroy inference platform
Influence over backend architecture and product direction
Hands-on work on hard problems in distributed systems, real-time streaming, and LLM serving
A small, senior team with fast execution and low bureaucracy
Competitive salary with performance-based incentives
24 days annual leave plus public holidays
Health insurance
Modern office in Yas Creative Hub
NICE TO HAVE
Firecracker, gVisor, Kata Containers, KVM or QEMU; Rust
vLLM, SGLang or llm-d in production; GPU scheduling or sharing (MIG, Kueue, HAMi)
Usage billing for APIs; Envoy or another proxy
RDMA or InfiniBand
Bare-metal or on-prem operations; multi-node training infrastructure
Job details are sourced from the employer's original posting.
Open job postingAbout the company
Aleroy provides AI-powered solutions for enterprise automation and workflow optimization.