Senior AI Engineer - Observability

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior AI Engineer - Observability based in United States. This is a hands-on senior engineering opportunity focused on making AI-powered products reliable, measurable, and production-ready. You’ll build and improve LLM, RAG, semantic search, and agentic experiences while helping define how AI quality is measured. A major focus will be designing evaluation pipelines, golden datasets, regression testing, and meaningful quality gates. You’ll also own observability across AI systems, tracking quality, latency, cost, reliability, safety, and real-world user feedback. Working closely with engineering, product, QA, security, and business teams, you’ll turn production signals into actionable improvements. You’ll help establish reusable standards, tooling, and engineering practices that enable teams to ship AI faster and with greater confidence. This role is ideal for an experienced software engineer who combines strong technical depth with product judgment and a rigorous quality mindset.