About the role
The Inference team turns trained models into a service. We own the serving stack from request admission to token streaming: batching, KV-cache management, kernels, routing and the autoscaling that keeps accelerators busy without breaking latency targets. Every percentage point of utilisation we recover is capacity that does not have to be bought.
Clexerl's own products sit on this stack. Clexerl Go agents make long chains of tool-heavy calls, long-memory features replay months of context, and multimodal inputs arrive as large, uneven payloads. Those workloads stress serving in ways a chat benchmark does not, and the team is where those pressures get absorbed and solved.
We work from profiles and load tests, not intuition. Changes ship behind canaries with latency and quality gates, and every incident ends in a written post-mortem that someone owns.
What you’ll do
- Build and tune continuous batching, paged KV-cache allocation and prefix caching to raise throughput while holding tail latency inside service-level objectives.
- Implement and evaluate speculative decoding, quantised serving and other techniques that cut cost per token without moving model quality.
- Design request routing and scheduling across heterogeneous accelerator pools, including prefill and decode disaggregation for long-context traffic.
- Profile production traffic end to end and find the stalls, whether in kernels, host code, networking or queueing.
- Build load-shedding, autoscaling and graceful-degradation paths so demand spikes slow us down rather than take us down.
- Work with research to land new architectures in the serving path with numerics checked against reference implementations.
What you’ll bring
- Several years building high-throughput, latency-sensitive services in C++, Rust, Go or Python, with strong habits around profiling and benchmarking.
- Working knowledge of transformer inference: attention variants, KV-cache memory behaviour, batching strategies and where decode is bound by memory bandwidth.
- Experience reading GPU or accelerator profiler output and reasoning about occupancy, memory traffic and host-device synchronisation.
- Comfort with distributed systems basics such as backpressure, retries, consistent hashing and failure domains.
- A record of shipping performance work that held up in production, with the measurements to prove it.
- Clear written communication, especially when explaining a trade-off to people who own the other side of it.
Nice to have
- Experience writing or modifying GPU kernels in CUDA, Triton or a comparable language.
- Familiarity with tensor and expert parallelism applied to serving large mixture-of-experts models.
- Background in capacity planning or cost modelling for accelerator fleets.
- Contributions to an open-source inference engine or serving framework.
Work you might take on
- 01
Introduce prefix-aware routing so agent sessions with shared system prompts land on replicas that already hold the cached context.
- 02
Cut time to first token for long-context requests by splitting prefill and decode across separately sized accelerator pools.
- 03
Build a replay harness that reruns sampled production traffic against candidate builds and flags latency or output regressions before rollout.
- 04
Roll out low-precision weight and KV-cache formats behind an eval gate, with automatic rollback if quality metrics drift.
Compensation
$300K – $480K
The US base salary range for this role. Every offer also includes equity in Clexerl and, when it spins out, in the venture you help build. We tell you the reasoning behind your number.
Benefits
Health, fully covered
Medical, dental and vision for you and your dependants, premiums paid in full.
Two kinds of equity
Equity in Clexerl, plus a stake in the venture you help build when it spins out.
22 weeks parental leave
Fully paid, for every parent, however your family arrives.
Rest that sticks
Flexible time off with a four-week minimum we actually check on, plus a company-wide week in December.
Learning and compute
A yearly budget for courses, conferences and books, and compute credits for side projects.
401(k) with a 4% match
Matched from your first paycheque and vested immediately.
How we hire
Apply 5 minutes
Send links to what you have built and a few lines on why this role. No cover letter needed.
Intro call 30 minutes
A conversation with the hiring manager about the work, the team and what you want next.
Work sample 2–4 hours
A paid take-home or a pairing session on a real-shaped problem. You pick the format.
Studio day One day
Meet the team, go deep on your work and ours. Remote or in person.
Offer Within 5 days
A written offer with the full range, the equity and the reasoning behind both.
Opens an email with a short template. Links beat résumés.