Overview

Senior Data Engineer – Front Office Commodity Trading

London, 3 days in the office (min)

Key Skills: Python, PySpark, Databricks, Data Science/ML

What you’ll do

• Build and optimize scalable data pipelines and Delta Lake architectures in Databricks on AWS
• Partner with data scientists and quants to deliver ML-ready datasets and production-grade AI pipelines
• Implement CI/CD, MLOps, and observability best practices using MLflow and Terraform
• Shape the Front Office data warehouse, ensuring speed, reliability, and accuracy
• Collaborate globally across tech and trading to drive innovation and automation

What you bring

• 8+ years building large-scale distributed data platforms
• Deep expertise in Databricks, Apache Spark, PySpark, Delta Lake, and Unity Catalog
• Strong Python, SQL, AWS, and Terraform skills
• Experience with DevOps/MLOps, and ideally exposure to streaming (Kafka) and real-time analytics
• Proven delivery in high-performance or trading environments

Seniority level

• Mid-Senior level

Employment type

• Contract

Job function

• Information Technology

Industries

• Oil and Gas and Securities and Commodity Exchanges