Overview

Bibby Financial Services has an exciting opportunity available for a reliable Senior Databricks Engineer to join our team on a full time, permanent basis. You will work in a hybrid role, with a competitive salary of £60,000 - £70,000 per annum. This role can be based in any of our UK offices.

Responsibilities

• Understand the business / product strategy and supporting goals to ensure data interpretation aligns
• Provide technical leadership on breaking down initiatives into appropriately sized features, epics and stories; set standards and drive quality and consistency in solution delivery
• Collaborate with the Data Architect to design our data architecture and translate into a build plan
• Lead the build and maintenance of scalable data pipelines and ETL processes for data integration and analytics from various sources, including cloud storage, databases and APIs
• Deliver large-scale data processing workflows (ingestion, cleansing, transformation, validation, storage) using best practice tools and techniques
• Collaborate with BI Product Owner, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs
• Optimize and tune data processing systems for performance, reliability and scalability
• Implement data quality and validation processes to ensure data accuracy and integrity
• Operate an agile CI/CD environment within Azure DevOps, contributing to Sprint cycles, code deployment, version control and development practices
• Develop and maintain data models, schemas, and documentation
• Monitor and troubleshoot data pipeline issues and ensure timely resolution
• Stay up-to-date with the latest industry trends in data engineering and recommend improvements
• Ensure adherence to BFS Governance processes
• Provide line management and technical leadership to a small team of Data Engineers
• Collaborate with the Data Architect to design and implement data architecture and build plans
• Lead the build and maintenance of scalable data pipelines and ETL processes across diverse data sources
• Deliver large-scale data processing workflows using best practice tools and techniques

Qualifications

• Significant Databricks experience, including Unity Catalog
• Terraform and infrastructure as code
• Proficiency in Python, Spark, SQL
• Strong experience with SQL databases
• Expertise in data pipelines and workflow tools (e.g., Apache Airflow, ADF)
• Experience with cloud platforms (Azure preferred) and related data services
• Excellent problem-solving skills and attention to detail
• Inclusive and curious mindset, eager to build knowledge and understanding
• Strong communication and collaboration skills
• Experience of Waterfall and Agile delivery methodologies

Benefits

• Private healthcare for you and your family
• Company pension scheme
• Flexible benefits (gym membership, technology, health assessments, etc.)
• Access to an online wellbeing centre
• Range of discounts from many businesses
• 25 days holiday, increasing with service, with options to buy or sell more
• Electric Vehicle/Plug-in Hybrid Vehicle (EV/PHEV) scheme

Why us?

We’ve supported small and medium-sized enterprises (SMEs) since 1982, and today we support more than 9000 businesses worldwide. We help businesses grow and thrive in domestic and international markets. We’re in the business of relationships and pride ourselves on brilliant people, a fast-paced and varied environment, ownership, flexibility, and delivering for our clients. We empower colleagues to be themselves and grow, and we celebrate success. There’s no place quite like BFS.

If you would like to join us, please click ‘Apply’ today to be considered as our Senior Databricks Engineer. This role is hybrid and can be based in any of our UK offices.

How to apply details

We reserve the right to close applications early. No agencies, please.

Job details

• Seniority level: Mid-Senior level
• Employment type: Full-time
• Job function: Information Technology