Research Scientist / Engineer, Inference

San Francisco | Permanent

DeepRec has partnered with a frontier AI research lab that designs its own model architectures from the ground up. Serving cost is treated as a research question here, so inference sits at the centre of the work rather than the end of it.


The Opportunity


You'll own latency and throughput across two serving stacks. One handles production traffic for customers. The other powers test-time compute and RL rollouts for the research team, so every gain you make speeds up the research itself.
 

What You'll Do

  • Cut latency and raise throughput for model serving at very high concurrency
  • Speed up RL rollout generation and scale test-time compute for research
  • Tune batching, caching, load balancing and parallelism across the serving stack
  • Write and profile GPU kernels in CUDA and Triton
  • Apply quantization, distillation, speculative decoding and low-precision numerics
  • Shape model and hardware co-design with serving cost in view from day one
  • Own benchmarking, testing and reliability for every inference service

What You'll Bring


Essential
  • Production experience with vLLM, TensorRT-LLM or similar at high concurrency
  • Hands-on continuous batching and KV cache management
  • Quantization for inference in production
  • Profiling and performance analysis with CUDA and Triton
Desirable
  • Speculative decoding or multi-node serving
  • Performance tuning on H100 or B200 hardware
  • Kubernetes and production infrastructure
We encourage you to apply even if you don't meet every requirement. The right mindset matters as much as the right CV.

 

 

Apply Now


If you've spent your career making models fast, this is a place where that work shapes the research agenda. Send your CV and we'll share the full brief in a confidential conversation