Senior/Lead Data Engineer

Data Engineers – Global Investment Firm | New York & London

Build the data platform behind trading, research and risk at a top-tier investment firm.

My direct client, a premier global investment firm is investing heavily in its data. The CTO has a big mandate to modernise the stack and build a more robust platform, and the team is hiring urgently. The team is small and growing, so your work will be visible and you'll have a direct say in the architecture. I placed a key person on this team earlier this year who is doing well and loves the environment and work.

? New York: four openings. One Lead Data Engineer and three mid/senior individual contributors. ? London: one mid/senior individual contributor, working closely with the New York team.

What you'll work on

• Data pipelines for trading, alpha generation, research, risk and accounting


• Golden-source datasets the whole firm relies on: security master, account master and price master


• Shared Python libraries for data APIs, logging and core functions


• The firm's AI tools, including LLM search and summarisation of financial documents and market commentary


• Data quality, observability and governance using modern tools


• Close work with quants, traders, risk managers, the Chief Data Officer and the Head of Development

Must-haves

• Hands-on experience with both Snowflake and Databricks


• Excellent Python (including PySpark) and SQL, plus Spark, dbt and AWS


• 3+ years of development experience, with at least 2 years focused on data engineering


• Deep ETL/ELT experience, strong data modelling, and SQL performance tuning (PostgreSQL or SQL Server)


• Docker and Kubernetes in cloud environments


• Some finance experience. A trading firm or hedge fund is ideal, or another fast-paced financial services firm.


• A degree in Computer Science or a related field

Nice to have

• A software engineering background as well as data engineering. The hiring manager values varied technical experience.


• Data catalogue and data lineage experience (for example OpenMetadata or Great Expectations)


• LangChain/LangGraph, vector databases and RAG


• Market data knowledge across equities, fixed income, futures and options


• FastAPI, Dask, or building BI dashboards

Why this role

• Real modernisation work with backing from a new CTO


• A small team where you'll have a lot of influence


• Bonus buy-outs possible for the right candidates

Process: a Zoom with the hiring manager, a Python-focused take-home on CoderPad, then a half-day onsite that includes a live coding session.

? Onsite 4 days a week in both New York and London

⚠️ Please note:

• Unfortunately, our client can't offer visa sponsorship. Candidates must be authorised to work in the US or UK without sponsorship, now and in the future.


• No third-party recruiters or agencies, please. We're only considering direct applicants.