Senior Data Engineer - Hybrid in London - Outside IR35

Overview

Our client is expanding their core data engineering function and hiring two Senior Data Engineers to strengthen a modern, scalable data platform. Both will be Databricks specialists, with one role focused on deep technical delivery and the other combining hands-on engineering with team leadership and upskilling responsibilities.

Key Responsibilities

Build and maintain Databricks Delta Live Tables (DLT) pipelines across Bronze ? Silver ? Gold layers, ensuring quality, scalability, and reliability.

Develop and optimise Spark (PySpark) jobs for large-scale distributed processing.

Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput.

Use Terraform and CI/CD pipelines (GitHub Actions or similar) to manage infrastructure and automate deployments.

(For leadership-focused role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team.

What's Expected

Proven experience deli...