Senior Data Engineer

Senior Data Engineer - Markets & AI Data Platform London | Hybrid £120,000–£140,000 base We’re hiring a Senior Data Engineer to join a specialist financial markets business building a new generation of global macro research, quantitative and AI-driven investment tools. The business brings together experienced macro strategists, quantitative researchers and engineers with backgrounds across leading hedge funds, investment banks and policy institutions. Their ambition is to combine decades of market expertise with quantitative models, alternative datasets and AI to surface investment insights and ultimately turn those insights into actionable global macro trade ideas. The data platform underneath that is critical. The role You’ll take ownership of the data engineering layer supporting the firm’s quantitative models, AI infrastructure and client-facing products. There is already a Quant Developer internally focused on productionising models, integrating them into Databricks and building much of the AI infrastructure. Your role is to complement that work by making sure the data feeding those models is reliable, appropriate, well-governed and production-ready. Today, multiple teams contribute data into the platform. As the Senior Data Engineer, you’ll become the person responsible for bringing structure, ownership and engineering discipline to that process. This is as much about understanding the data as it is about moving it. What you’ll own You’ll build, improve and operate production data pipelines covering datasets such as: - Market and pricing data - Tick and time-series data - Flow and positioning data - Economic and macro datasets - Research and alternative data - News and internal datasets - Proprietary financial datasets You’ll be responsible for making sure those datasets are suitable for downstream use by quantitative models, researchers, AI systems and clients. That includes: - Building and operating robust ETL/ELT pipelines - Designing reliable bronze, silver and gold data layers - Creating data quality and validation controls - Monitoring pipeline health and resolving production issues - Managing lineage, provenance and metadata - Instrument and ticker mapping - Building model-ready datasets for quants and AI systems - Implementing permissions and entitlements - Ensuring different clients only access the data and models they are entitled to - Assessing new data sources before they enter production - Working with alternative vendors where traditional sources cannot be used - Improving reliability as ingestion becomes increasingly frequent Current ingestion can be around every 15 minutes across certain datasets, with the expectation that this will become more real-time as the platform develops. Financial markets data This is an important part of the role. The team is not simply looking for someone who can build a technically sound pipeline. They need someone capable of understanding whether the underlying financial data makes sense. You should be comfortable working with complex market datasets and thinking about questions such as: - Is this source appropriate for the model? - How reliable is the historical data? - How should instruments be mapped across different vendors? - What happens when two sources disagree? - How should missing or stale data be handled? - What alternative sources can be used where major market-data vendors are unavailable? Experience with market data, tick data, pricing, flow, positioning, reference data or other financial time-series datasets would therefore be highly valuable. Technical environment You should have strong hands-on experience with: - Python - SQL - Production ETL/ELT pipelines - Databricks or similar modern data platforms - Cloud data architecture - Data quality and monitoring - Time-series or complex datasets Experience with any of the following would be useful: - Azure - Databricks - Delta Lake - Snowflake - kdb+ - Financial market data platforms - Data lineage / observability tooling - Data entitlements and permissions The exact technology matters less than your ability to build reliable systems and understand the data flowing through them. Working environment This is a small, highly experienced and very flat team. You’ll work directly with: - Quant Developers - Quant Researchers - Macro Strategists - AI Engineers - Product and commercial stakeholders There are few layers between identifying a problem and solving it. You’ll be expected to communicate well, challenge assumptions and take genuine ownership rather than simply work through a backlog of predefined engineering tickets. Why this role is different The team is attempting something genuinely ambitious. They are bringing together decades of global macro expertise across strategists, economists, quantitative researchers and algo traders, combining those insights and models within one technology platform. The longer-term goal is to create a system capable of continuously understanding global macro markets, surfacing investment opportunities and turning those insights into trade ideas for institutional clients. That ambition only works if the underlying data can be trusted. The person joining this team will therefore have a significant influence over the reliability, credibility and scalability of the entire platform. For a Data Engineer who wants to work close to markets, quants and AI and wants ownership rather than a narrow infrastructure remit this is a genuinely unusual opportunity.