Senior Software Engineer

About the role & what you’ll do Run a fleet of coding agents to build inference infrastructure in Rust and Python to power modern AI applications. You'll work with high-performance services for running AI models, metering tokens, processing images, audio and video, used by real teams in our open source K8s inference cluster and our serverless API product. This role is hands-on and end-to-end: from designing clean APIs to making them fast, observable and easy to operate. Tech stack - Python: Python 3.12+, FastAPI, Pydantic, asyncio, uv, pytest, ruff, mypy - Ecosystem: Kafka, VDBs (Qdrant, Redis, TopK), OpenTelemetry (Loki, Grafana, Tempo, Mimir, Prometheus), Infinity, Locust - Infrastructure: GCP (CloudRun, GKE), Kubernetes, Docker, Terraform, Github Actions, mise, Monorepo - Data: Ray, Polars, Pandas - Rust: Hands-on experience debugging, troubleshooting, and optimizing Rust code. - AWS (EKS): Experience with AWS, particularly Elastic Kubernetes Service (EKS). - GCP: Experience with Google Cloud Platform Soft skills - Strong ownership: able to take a problem end-to-end with minimal supervision. - Clear, concise communication in a distributed international team. - Reliable execution and ability to deliver within agreed timelines. Originally posted on Himalayas