Data Engineer

New Yesterday

.

We are seeking a highly capable and driven Data Engineer to join our dynamic data team. This role is pivotal in leading and initiating the migration of our data infrastructure from AWS to Microsoft Azure, shaping the future state of our data platform. You’ll work at the heart of the business, enabling data integration, transformation, and reporting across a range of complex data sources.

This is a strategic, hands-on role that requires someone who is not afraid to explore new technologies, drive change, and build future-proof data engineering solutions. You’ll collaborate with cross-functional teams, manage data integrations, and optimize the performance of databases and cloud environments in a high-growth, fast-paced setting.

YOUR RESPONSIBILITIES:

Cloud Migration Leadership

  • Play a key role in the execution of our data migration from AWS to Azure, working closely with the Solution Architect and Head of data to shape the technical approach and deliver scalable, cloud-native solutions.
  • Evaluate existing data infrastructure and propose Azure-native solutions aligned to business needs.
  • Define and lead the migration roadmap, collaborating with internal stakeholders and third parties.
  • Advocate for innovation and drive adoption of Azure services, standards, and tools across the team.
  • Apply and champion Azure Medallion architecture principles in future-state design.

    • Data Ingestion & Integration

  • Design and implement ingestion pipelines into Azure (Data Lake, Synapse, Blob Storage, Azure SQL).
  • Ingest structured, semi-structured, and unstructured data from APIs, FTP/SFTP, cloud platforms, and file-based sources.
  • Manage data integrations with suppliers, including file transfers and API-based solutions.
  • Use Power Automate (Flow) to automate tasks like copying files from SharePoint or emails to SFTP.
  • Provide third-line support for code issues and existing workflows, lead resolution of data failures and integration problems.
  • Collaborate with stakeholders to define integration requirements and implement robust solutions.

    • Data Transformation & ETL

  • Build, optimize, and maintain ETL/ELT pipelines using Azure Data Factory and other cloud-native tools.
  • Develop reusable, scalable transformation logic to enable analytics and reporting needs.
  • Apply data validation, cleansing, and lineage tracking throughout the pipeline lifecycle.
  • Support both existing and future ETL workloads with strong SQL and scripting capabilities.

    • Data Platform & Server Management

  • Revamp and support existing database designs to enable a scalable migration to Azure.
  • Introduce and enforce standards and governance around data structures and code.
  • Optimize T-SQL code and SQL Server performance before deploying into production.
  • Manage consistent data platforms across regions with a focus on reusability and modularity.
  • Monitor and control database file growth, indexing strategies, and performance tuning.
  • Develop scalable solutions based on both current and emerging business requirements.

    • Data Cloud Operations & Resilience

  • Lead initiatives in data cloud management, including usage monitoring and optimization.
  • Configure and test disaster recovery processes, ensuring business continuity through backup and restore exercises.
  • Archive or delete legacy data in line with governance and retention requirements.

    • Delivery & Documentation

  • Conduct data mapping exercises, analysing various sources and formats to design fit-for-purpose models.
  • Create and present complete logical data models to both technical teams and business stakeholders.
  • Participate in scoping future platforms and assessing legacy systems for migration to Azure.
  • Perform functional and systems analysis, producing supporting documentation (e.g., flowcharts, data diagrams).
  • Be the lead in discovery workshops, weekly stand-ups, and cross-functional solution delivery.

    • YOUR TECHNICAL SKILLS & EXPERERIENCE 

    Essential

  • 6–10 years’ experience working with SQL Server (including T-SQL development and performance tuning).
  • Proven experience designing ETL/ELT pipelines, ideally in Azure Data Factory.
  • Hands-on experience ingesting and transforming data in cloud environments (preferably Azure).
  • Familiarity with Azure Medallion architecture or similar layered data designs.
  • Integration experience with APIs, SFTP, cloud databases, and file systems.
  • Comfortable working with Power Automate, SharePoint, and Office integrations.
  • Basic knowledge of C# or PowerShell for custom data integration tasks.
  • Experience troubleshooting production systems and optimizing SQL workloads.
  • Strong understanding of database administration, performance, and scalability.
  • Cloud certification (e.g., Azure Practitioner or equivalent) is highly desirable.

    • Desirable

  • Experience working with notebooks and Python in Microsoft Fabric for data exploration, transformation, and pipeline development.
  • Experience with AWS services and data environments, especially helpful for transition planning.
  • Exposure to CI/CD, DevOps pipelines, and Git for data engineering workflows.
  • Familiarity with data governance, quality management, and security practices.

    • YOUR BEHAVIOURS & ATTRIBUTES

  • Technically curious and excited by the opportunity to learn and adopt new tools.
  • Naturally proactive and comfortable driving change in existing environments.
  • Calm and solution-oriented in high-pressure or production-critical scenarios.
  • Collaborative and open communicator who brings people on the journey.
  • Able to travel occasionally within Europe to support cross-territory platforms and teams.

    • LOCATION

  • This role is based in our West London Office including some remote working 
    • Apply
    Location:
    London
    Job Type:
    FullTime

    We found some similar jobs based on your search