Senior Data Engineer Python
Job Location:
Monthly Salary:
Posted on:
Vacancies:
Job Summary
Senior Data Engineer - Python
Location
London
Business Area
Marketing and Public Relations
Ref #
Description & Requirements
The Climate Data Utility Tech Team is actively searching for an experienced Senior Data Engineer to play a pivotal role in the design implementation enhancement and maintenance of scalable data pipelines for the Net-Zero Data Public Utility. These pipelines are essential for the Utilitys mission of providing open and accessible public good data through both the Climate Data Utility website and APIs.
A successful candidate will face the challenge of working with data originating from a wide array of sources each with its own formats fields and access protocols. Your responsibilities will encompass the full data lifecycle from the extraction of data from sources transforming it according to source and domain specific business logic and pushing it through the Utilitys ingestion process. Additionally you will be expected to implement data quality checks and validation procedures to ensure the accuracy and reliability of the data provided by the Utility.
Responsibilities:
* Design and develop scalable data pipelines using Python to read from APIs and structured files (Excel Parquet).
* Translate domain and source specific business logic into efficient code implementations to turn source data into usable structured data for downstream applications.
* Implement data transformations and structuring using tools like Pandas and Pydantic to ensure data quality consistency and adherence to business logic requirements.
* Collaborate with data scientists and analysts to support data-driven decision-making.
* Document data engineering processes including data lineage data dictionaries and system architectures
* Maintain best practices through code reviews version control and adherence to industry standards.
* Deploy production quality code through CI/CD pipelines into our cloud environment.
* Provide mentorship and guidance to junior team members fostering their growth and development in data engineering practices.
Qualifications
* 7 years of experience in data engineering or a similar role.
* Proficiency in Python programming.
* Experience with building and managing data pipelines.
* Knowledge of data warehousing and ETL processes.
* Excellent problem-solving and communication skills.
A successful candidate will face the challenge of working with data originating from a wide array of sources each with its own formats fields and access protocols. Your responsibilities will encompass the full data lifecycle from the extraction of data from sources transforming it according to source and domain specific business logic and pushing it through the Utilitys ingestion process. Additionally you will be expected to implement data quality checks and validation procedures to ensure the accuracy and reliability of the data provided by the Utility.
Responsibilities:
* Design and develop scalable data pipelines using Python to read from APIs and structured files (Excel Parquet).
* Translate domain and source specific business logic into efficient code implementations to turn source data into usable structured data for downstream applications.
* Implement data transformations and structuring using tools like Pandas and Pydantic to ensure data quality consistency and adherence to business logic requirements.
* Collaborate with data scientists and analysts to support data-driven decision-making.
* Document data engineering processes including data lineage data dictionaries and system architectures
* Maintain best practices through code reviews version control and adherence to industry standards.
* Deploy production quality code through CI/CD pipelines into our cloud environment.
* Provide mentorship and guidance to junior team members fostering their growth and development in data engineering practices.
Qualifications
* 7 years of experience in data engineering or a similar role.
* Proficiency in Python programming.
* Experience with building and managing data pipelines.
* Knowledge of data warehousing and ETL processes.
* Excellent problem-solving and communication skills.
Discover what makes Bloomberg unique - watch our podcast series for an inside look at our culture values and the people behind our success.
Required Experience:
Senior IC
About Company
Bloomberg is the world's primary distributor of financial data and a top news provider of the 21st century. A global information and technology company, we use our dynamic network of data, ideas and analysis to solve difficult problems every day. Our customers around the world rely on ... View more
