Senior Data Engineer
New Today
Overview
Engelhart was founded in 2013 by BTG Pactual Group as a commodities trading company. Our business model is “asset light” and highly diversified, giving us the ability to adapt effectively to changing market conditions. We have assembled successful multidisciplinary teams, leveraging advanced fundamental analysis with deep quantitative and weather research capabilities. Our activities are underpinned by strong risk management practices and by powerful technology and operational excellence. We have exceptional teams with diverse global backgrounds and decades of experience, and are driven by a highly collaborative culture, across products and competencies.
In 2024, Engelhart acquired Trailstone, a global energy trading and technology company. The acquisition provides us with new expertise, analytics and proprietary technology which is being used to provide risk management and optimisation services to help maximise the value of our clients’ renewable power. The acquisition also expanded Engelhart’s capabilities into physical natural gas across North America, a critical fuel to support the energy transition.
Our talented and experienced individuals work together according to its four company values: be bold, be collaborative, be proactive, be your best.
About the Role
As a member of the Tech Data, AI & UX team, the Senior Data Engineer is responsible for preparing data for analytical or operational uses, typically building and maintaining data pipelines to pull together information from different sources; integrating, consolidating and cleansing data; and structuring it for use in individual analytics applications in a ready-to-use form for data scientists who are looking to run queries and algorithms against the information for predictive analytics, systematic trading, machine learning and data mining purposes.
The Technology Data Analytics team develops data solutions to support business decisions by providing raw / pre-processed data to Data Scientists and Traders. This is achieved with agile methodology and a lot of teamwork. The team focuses on data quality, performance and self-service, and embraces cutting-edge technology in Data Science, Cloud Computing, Containerization and AI.
This role requires a highly skilled individual with technical depth, strong problem-solving capabilities, and the ability to collaborate effectively in a dynamic and fast-paced environment. This will be a full-time role, owning the following responsibilities:
- Working closely with the business, providing data support to its initiatives and projects, collecting requirements and discussing solutions with Data Analysts, Quant Developers, Researchers.
- Ingesting structured or unstructured data from new sources through scraping scripts, services, jobs, crawlers, ETL and streaming.
- Data cleaning, enrichment or transformation using appropriate tools for quick delivery and greater reuse.
- Continuously developing the data framework of the company, with the aim of streamlining onboarding, distribution and visualization.
- Safe and efficient persistence of data via several types of storage, such as DBs, warehouses, and data lakes in different formats.
- Internally distributing data through in-house APIs, Python packages, and/or direct DB queries.
- Maintaining existing data pipelines and databases.
- Supporting the automation of post-processing routines when required, including prediction models and visualization.
- Supporting Data Scientists in the use of common internal libraries in Python, cloud resources, and access to existing data.
- Knowledge retention and maintenance of a knowledge base on data sources and pipelines.
- Developing new visualizations using data visualization platforms.
About You
The following education, experiences and skills are essential for application:
- Background in Engineering, Computer Science, Systems Analysis, Data Science, Information Systems, or substantive commensurate experience.
- Experience with Commodities or Financial Markets.
- Experience interfacing with traders and quant developers.
- Excellent communication and organizational skills.
- Proven ability to communicate data concepts to both technical and business users, influencing decision making.
- Proven ability to problem solve and resolve conflict in a work environment.
- Proven, extensive track record of project ownership and driving results end to end.
- Advanced proficiency in scripting languages such as Python for data crawling, parsing, etc.; mastery of the Pandas library.
- Significant experience working with Analytical / Time-series databases (ClickHouse is strongly preferred; Redshift will also be considered).
- Substantive working experience with Docker Containers.
- Substantive working experience with DevOps or GIT source code repository.
- Use of IaC and CI/CD with AWS (eg. CloudFormation, CDK).
- Proactive, pragmatic problem-solving mentality, with a strong bias for action.
- Positive, team-oriented mindset.
- Shows ownership and accountability in all aspects of work.
- A proven ability to continuously learn and challenge the status quo.
In addition, the following skills are not required, but highly desirable for this role:
- Exposure to integration tools such as Rest API, Web API, Entity Framework, SSIS.
- Knowledge of other common AWS resources (S3, Lambda, Athena, EMR, Faregate, Kinesis, EC2, API Gateway, CloudWatch).
- Big Data processing tools & formats such as Apache Spark, Dask, Databricks, Hadoop, MapReduce, Parquet, Presto, Sqoop, Hive, HBase.
- Experience with RAD / Agile methods.
- Understanding of AWS Cloud Architecture and its Security (e.g., IAM, Cognito).
- Experience of visualization libraries such as Plotly.
- Knowledge of web formats (e.g., HTML, JSON) and experience with web scraping.
- A combination of relevant statistical / math knowledge.
- Curiosity in new technologies surrounding AI, ML, IoT, etc.
Who You Are
- You are customer obsessed.
- You have strong communication skills.
- You believe in the power of a team, diversity of thought, and unity of action.
- You always act with integrity, keep your commitments, and practice transparency.
- You think boldly, lead with courage, and demand the highest standards.
- You drive for results and act like an owner.
What we offer
- Competitive compensation and participation in Engelhart’s discretionary bonus plan.
- 25 days of annual holiday entitlement, excluding UK public holidays.
- Robust benefits package such as medical, dental, life insurance, generous pension contribution, and supplemental benefits.
- Eligibility to receive external and internal training in accordance with our Training & Development Policy.
We believe in inclusivity and are therefore dedicated to ensuring all employees – across gender identity, race, ethnicity, sexual orientation, religion, life experience, background and more – feel welcome and included in the company. We promote diversity because we believe it is essential to our ability to think holistically.
- Location:
- City Of London
- Job Type:
- FullTime
- Category:
- IT & Technology
We found some similar jobs based on your search
-
New Today
Senior Data Engineer in London - Harrison Holgate
-
City Of London, England, United Kingdom
Key skills and responsibilitiesProven experience in data engineering and data platform developmentStrong programming skills in Python, Java, Scala, or similarAdvanced SQL and deep knowledge of relational databasesHands-on experience with ETL tools an...
More Details -
-
New Today
Senior Data Engineer
-
City Of London, England, United Kingdom
About the Business At ICIS, our mission is to optimize the world's resources. We help companies make strategic, sustainable decisions by bringing transparency to markets across the world. We create a comprehensive view of commodities markets, providi...
More Details -
-
New Today
Senior Data Engineer
-
City Of London, England, United Kingdom
OverviewSenior Data Engineer (up to £63,250 per annum)Immediate is home to some of the biggest and most loved consumer brands in the UK, including Radio Times, Good Food and BBC Gardeners World magazine. Our trusted, quality content reaches millions ...
More Details -
-
New Today
Senior Data Engineer
-
City Of London, England, United Kingdom
OverviewOur MissionAt Ophelos, we believe in a different way to deal with debt. One that puts empathy and understanding front and centre, approaching our customers as individuals - no matter what they're going through.For too long, people in debt hav...
More Details -
-
New Today
Senior data analytics engineer
-
City Of London, England, United Kingdom
This job is brought to you by Jobs/Redefined, the UK's leading over-50s age inclusive jobs board.This is a hybrid position with one of our offices in London, Bath or Edinburgh.About usSeccl is the Octopus-owned embedded investment platform that's on ...
More Details -
-
New Today
Senior Data Engineer
-
United Kingdom
About StarcompliaceStarCompliance is on a mission to make compliance simpleandeasy. Trusted globally by enterprise financialStarCompliance is on a mission to make compliance simpleandeasy. Trusted globally by enterprise financial institutions, the us...
More Details -