Senior ML Inference / Systems Engineer

*hiring on behalf of our client Title: Senior ML Inference / Systems Engineer Time: Full-Time Location: Remote (Germany) Compensation: $250,000 - $300,000 USD / year + competitive equity The Company Our client is a high-growth research lab and inference provider focused on real-time AI for consumer-facing applications. They build first-party speech models, serve LLMs, and operate the infrastructure behind modular APIs designed for high-volume, real-time workloads. Their platform serves over 10 trillion LLM tokens monthly to hundreds of millions of users across companions, healthcare, education, and media. Backed by top-tier venture capital firms, they collaborate with leading technology partners to power sub-second multimodal experiences at global scale. Role Overview Our client is seeking an experienced Senior ML Inference / Systems Engineer to optimize, scale, and maintain high-throughput, low-latency machine learning serving systems. In this role, you will take ownership of the full model deployment lifecycle, from containerizing and profiling models to implementing cutting-edge acceleration techniques across distributed GPU clusters. You will work in a fast-paced, high-impact environment focused on solving latency, throughput, and distributed systems challenges in production. Key Responsibilities • Design, build, and optimize high-performance inference pipelines for large-scale production workloads • Implement advanced model acceleration techniques including quantization, distillation, continuous batching, paged attention, and speculative decoding • Profile and optimize low-level code to maximize performance and efficiency across NVIDIA GPUs • Scale distributed inference infrastructure using modern serving frameworks, Kubernetes, Ray, and custom load-balancing solutions • Containerize models and collaborate closely with research teams to deploy stable, high-throughput systems into production • Troubleshoot and resolve complex system bottlenecks to ensure high availability and sub-second latency across multi-GPU and multi-node setups • Write clean, high-performance systems code and contribute to core engineering architecture decisions Qualifications • 5+ years of experience as an ML inference engineer, systems engineer, or senior backend engineer working with high-performance production systems • Deep technical expertise in modern inference serving frameworks such as vLLM or TRT-LLM • Proficiency in systems programming and GPU acceleration using C++, CUDA, Rust, or highly optimized Python • Hands-on experience deploying and managing distributed systems at scale using Kubernetes, Ray, and Docker • Demonstrated background implementing model acceleration strategies such as caching, quantization, distillation, or speculative decoding • Strong foundation in systems engineering, low-latency architectures, and multi-GPU/multi-node scaling • Bachelor’s, Master’s, or PhD in Computer Science, Mathematics, Physics, or equivalent practical systems engineering experience • Excellent problem-solving skills with a strong bias for shipping stable, high-impact solutions in ambiguous environments