Data Engineer

Location: London

Grade: GG10 – GG12

BAE Systems Digital Intelligence is home to 4,500 digital, cyber and intelligence experts. We work collaboratively across 10 countries to collect, connect and understand complex data, enabling governments, nation states, armed forces and commercial businesses to unlock digital advantage in the most demanding environments.

Key Responsibilities

• Design, implement, and optimize robust ETL/ELT data pipelines using Apache Airflow as an orchestration tool.
• Build and manage data ingestion processes from internal systems and APIs using tools like Kafka, Spark, or AWS native services.
• Develop and maintain data lakes and data warehouses (e.g., AWS S3 + Redshift).
• Ensure data quality, validation, and governance using automated testing tools.
• Monitor and debug data workflows, implement logging, alerting, and performance tuning.
• Handle schema evolution, partitioning, and efficient data modelling.
• Collaborate with DevOps to manage CI/CD pipelines for data deployments and ensure version control of DAGs and configs.
• Apply best practices in security and compliance, including data encryption, access control (IAM), and regulatory standards (GDPR).

Required Technical Skills

• Proficient in Python and SQL for data processing.
• Hands‑on experience with Apache Airflow, writing Airflow DAGs and configuring Airflow jobs.
• Experience with AWS cloud platform and services such as S3 and Redshift.
• Familiarity with big data processing using Apache Spark.
• Knowledge of data modelling, schema design, and partitioning strategies.
• Understanding of batch vs. streaming data paradigms and tools such as Kafka.
• Experience working with Docker or Kubernetes.

Benefits

As well as a competitive pension scheme, BAE Systems also offers an employee share plan, an extensive range of flexible discounted health, wellbeing and lifestyle benefits including a green care scheme, private health plans and shopping discounts – you may also be eligible for an annual incentive.

Why BAE Systems?

This is a place where you’ll be able to make a real difference. You’ll be part of an inclusive culture which values diversity, rewards integrity and merit, and where you’ll be empowered to fulfil your potential. We welcome candidates from all backgrounds and particularly from sections of the community who are currently under‑represented within our industry, including women, ethnic minorities, people with disabilities and LGBTQ+ individuals.

Inclusive Recruitment & Diversity

We also want to make sure that our recruitment processes are as inclusive as possible. If you have a disability or health condition, please speak to your recruiter about potential reasonable adjustments.

Security & Vetting

Many roles at BAE Systems are subject to both security and export control restrictions. Applicants must achieve baseline personnel security standard and may require higher levels of national security vetting, typically with 5–10 years of continuous residency in the UK.

Life at BAE Systems Digital Intelligence

We are embracing Hybrid Working. This means you and your colleagues may be working in different locations… Hybrid Working allows increased flexibility in when and where we work, helping us to balance our work and personal life more effectively and enhance well‑being. Diversity and inclusion are integral to the success of BAE Systems Digital Intelligence, with a culture that welcomes employees with varying perspectives, skills, life experiences and backgrounds.