Data Platform Engineer
New Yesterday
Overview
At Octopus we've developed a data platform that is used by all our businesses around the world. The platform empowers users with self-service data analytics and automates our data processing workflows, from simple ETL jobs to ML training and prediction.
The data platform team works on a range of initiatives from natural language understanding of customer communications to processing billions of smart meter readings every day to support customised and market-leading smart energy tariffs.
We're looking for an Engineer to join the team to help us build and maintain our platform, pipelines and reporting tools. This is a fantastic opportunity to work with us on data problems that genuinely move us closer to Net Zero and support the energy transition. We employ software engineering best practices to design, test, and deploy our data platform and services.
Responsibilities
- Build and optimise the platform used by the data team to transform raw data into business insights
- Scope and deliver new data engineering projects in collaboration with business stakeholders
- Develop and deploy AI tools to help build out our ever-growing AI requirements and use cases
- Research and implement new tooling where appropriate, whether it's developed in-house or sourced externally
- Maintain and expand our existing data infrastructure and tools
- Consult the wider business on engineering best practices and stay up to date with trends within the profession
- Support the internationalisation of our data infrastructure as we continue to grow globally
What you\'ll have
- Primarily, we want our engineers to have a passion for writing high quality code and building lean processes
Essential
- Python
- AWS
- SQL (Spark/Postgres preferably)
Areas for Growth
- Airflow
- Experience using CLI tools
- Docker/EKS
- Spark
- DBT
- Circle CI
- The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve and deliver pragmatic solutions
Our Tech Stack
- Python as our main programming language
- Circle CI for continuous deployment
- Databricks for our data lake platform
- Parquet and Delta file formats on S3 for data lake storage
- Terraform on Spacelift for our infrastructure definition and deployment
- Kubernetes for data services and task orchestration
- Airflow for job scheduling and tracking
- Postgres / Aurora for our relational databases
- Spark for data processing
- dbt for data modelling
- SparkSQL for analytics
- Streamlit for data applications
Our process usually takes up to 4 weeks, but we'll always do our best to flex around what works for you. Along the way, you'll chat with our recruitment team and your Recruiter will help you throughout different stages. Got any burning questions before then? Drop us a message at hiring@octoenergy.com and we'd love to help!
If this sounds like you then we'd love to hear from you.
Are you ready for a career with us? We want to ensure you have all the tools and environment you need to unleash your potential. Need any specific accommodations? Whether you require specific accommodations or have a unique preference, let us know, and we'll do what we can to customise your interview process for comfort and maximum magic!
Studies have shown that some groups of people, like women, are less likely to apply to a role unless they meet 100% of the job requirements. Whoever you are, if you like one of our jobs, we encourage you to apply as you might just be the candidate we hire. Across Octopus, we\'re looking for genuinely decent people who are honest and empathetic. Our people are our strongest asset and the unique skills and perspectives people bring to the team are the driving force of our success. As an equal opportunity employer, we do not discriminate on the basis of any protected attribute. Our commitment is to provide equal opportunities, an inclusive work environment, and fairness for everyone.
Job details
- Seniority level: Not Applicable
- Employment type: Full-time
- Job function: Engineering and Information Technology
- Industries: Utilities and Environmental Services
#J-18808-Ljbffr
- Location:
- London, England, United Kingdom
- Salary:
- £125,000 - £150,000
- Job Type:
- FullTime
- Category:
- IT & Technology
We found some similar jobs based on your search
-
New Today
Lead Software Engineer - Data Platform in London - LinuxRecruit
-
London, England, United Kingdom
-
£150,000 - £200,000
- IT & Technology
Core Stack Our core stack focuses on Python, TypeScript, GraphQL, AWS, and Kubernetes. We encourage breaking the mold and exploring new technologies if you believe they are superior. Python TypeScript GraphQL AWS Kubernetes Encourage exploring new t...
More Details -
-
New Today
Data Platform Engineer
-
London, England, United Kingdom
-
£125,000 - £150,000
- IT & Technology
Overview Data Platform Engineer role at Sonnedix. We are a fast-growing global renewables power producer building a bright future. The Data Platform Engineer will support the development of Sonnedix’s Data Platform, leveraging cloud and big data exp...
More Details -
-
New Today
Cloud & AI Solution Engineer – Data Platform
-
London, England, United Kingdom
-
£125,000 - £150,000
- IT & Technology
Overview Join to apply for the Cloud & AI Solution Engineer – Data Platform role at Microsoft . Are you insatiably curious, deeply passionate about the realm of databases and analytics, and ready to tackle complex challenges in a dynamic environm...
More Details -
-
New Today
Data Platform Engineer
-
London, England, United Kingdom
-
£125,000 - £150,000
- IT & Technology
Overview MoonPay is building infrastructure to power the decentralized economy. We facilitate buying, selling, and trading crypto using everyday payment methods and provide tools to send, receive, and manage stablecoins. We operate in a licensed, re...
More Details -
-
New Yesterday
Data Platform Engineer
-
London, England, United Kingdom
-
£125,000 - £150,000
- IT & Technology
Overview At Octopus we've developed a data platform that is used by all our businesses around the world. The platform empowers users with self-service data analytics and automates our data processing workflows, from simple ETL jobs to ML training a...
More Details -
-
New Yesterday
Quality Assurance Engineer (Data Platform)
-
London, England, United Kingdom
-
£125,000 - £150,000
- Engineering
Quality Assurance Engineer (Data Platform) The Company Founded in 2018, our client are revolutionising the insurance market and have grown year on year. They operate a remote set-up and the overall business are made up of self-managed and collaborati...
More Details -