About the Role

We’re looking for a skilled and experienced Lead Data Engineer on behalf of a London based client to help build and scale real-time data capabilities within a fast-moving digital organization. This is a senior-level, hands-on role with both technical ownership and leadership responsibilities, including the line management of engineers.

You’ll architect and deliver scalable data infrastructure and pipelines, enabling real-time analytics and decision-making across the business. Working closely with engineering, product, and analytics teams, you'll play a critical role in ensuring data is accurate, timely, and usable across the organization.

What You'll Be Doing

• Design, develop, and maintain real-time data pipelines that are scalable, efficient, and fault-tolerant
• Integrate data from multiple sources to support real-time insights and downstream applications
• Optimise data infrastructure for low-latency and high-throughput performance
• Lead and mentor data engineers, providing direction, support, and regular feedback
• Define and enforce best practices around data quality, security, and governance
• Collaborate with cross-functional teams to deliver solutions aligned with business goals
• Own monitoring, alerting, and logging processes to ensure pipeline stability and resilience
• Continuously improve data architecture, tooling, and workflows

What We're Looking For

• Strong track record of delivering real-time data pipelines and infrastructure
• Hands-on experience with distributed data processing using tools like Apache Kafka, Apache Spark Streaming, or Apache Flink
• Proficient in Python, Java, or Scala
• Deep understanding of SQL, NoSQL, and time-series databases
• Proven ability to optimise, troubleshoot, and scale data systems in production
• Experience with orchestration and deployment tools like Apache Airflow and Kubernetes
• Advanced expertise in cloud-based data engineering—Google Cloud Platform (GCP) experience is strongly preferred and highly valued
• Familiarity with BigQuery, Pub/Sub, Dataflow, and other GCP-native tools is a major advantage
• Line management or formal leadership experience—comfortable managing engineers and fostering team growth
• Excellent communication and collaboration skills, with a proactive, team-first attitude
• Comfortable working in a fast-paced environment with changing priorities