Senior Data Engineer

New Yesterday

Senior Data Engineer - Hybrid - Contract - Outside IR35 Our client is expanding their core data engineering function and hiring two Senior Data Engineers to strengthen a modern, scalable data platform. Both will be Databricks specialists, with one role focused on deep technical delivery and the other combining hands-on engineering with team leadership and upskilling responsibilities.
Key Responsibilities Build and maintain Databricks Delta Live Tables (DLT) pipelines across Bronze → Silver → Gold layers, ensuring quality, scalability, and reliability. Develop and optimise Spark (PySpark) jobs for large-scale distributed processing. Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput. Use Terraform and CI/CD pipelines (GitHub Actions or similar) to manage infrastructure and automate deployments. (For leadership-focused role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team.
What’s Expected Proven experience delivering end-to-end data pipelines in Databricks and Spark environments. Strong understanding of data modelling, schema evolution, and data contract management. Hands-on experience with Kafka, streaming architectures, and real-time processing principles. Proficiency with Docker, Terraform, and cloud platforms (AWS, GCP, or Azure) for scalable data infrastructure. Demonstrated ability to lead, coach, and elevate team capability, fostering best practices and knowledge sharing.
Apply
Location:
London, England, United Kingdom
Job Type:
FullTime

We found some similar jobs based on your search