Lead Data Engineer UK
Lead Data Engineer UK
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
CloudSmiths is looking for a Lead Data Engineer to support our UK customers.
This role is integral to the design development and maintenance of our data solutions ensuring scalability reliability and performance in our clouddriven ecosystem. The ideal candidate will possess strong technical expertise in data engineering coupled with a proven track record in leading technical teams and implementing datadriven projects in an Agile environment.
Key Responsibilities & Outputs:
- Build and maintain efficient ETL/ELT pipelines using tools such as Apache Airflow and PySpark.
- Develop database schemas dimensional models (Kimball/Inmon) and support data normalisation for relational and NoSQL databases.
- Participate in the development and maintenance of data warehouses data lakes and data lakehouses.
- Work with relational database systems such as Azure SQL PostgreSQL and Google Big Query as well as NoSQL systems like MongoDB and Google Firestore.
- Handle structured semistructured and big data file formats including Avro CSV Parquet ORC and Delta.
- Develop and maintain APIs for seamless data integration and workflows with an understanding of REST and microservices architectures.
- Oversee the maintenance and optimisation of the codebase using Git for version control.
- Implement integration testing and ensure highquality deliverables for all new data processing scenarios.
- Provide technical design and coding assistance to team members ensuring the successful delivery of project milestones.
- Assess and integrate new data sources into pipelines to meet evolving business needs.
Education Experience & Knowledge:
- Strong proficiency in Python and SQL (PostgreSQL or SQL Server preferred). Handson experience with Apache Airflow and PySpark.
- Familiarity with Databricks (preferred but not mandatory).
- Working knowledge of cloud platforms such as Azure GCP or AWS.
- Experience with data warehousing concepts dimensional modelling and database normalisation.
- Understanding of big data file formats such asAvro Parquet ORC and Delta.
- Proficiency in working with APIs REST and microservices architectures.
Education & Experience:
- Bachelors degree in Computer Science Data Science or related fields.
- 5 years of progressive experience in data engineering cloud computing and technology implementation.
- Experience managing multishore projects and working within cloud ecosystems (SaaS/PaaS).
- Proven experience leading technical teams and mentoring team members.
Why CloudSmiths Join us at CloudSmiths and make a mark in the world of technology!
Drive your career forward and be part of our success story! We offer a dynamic work environment competitive incentives and the opportunity to work with cuttingedge technology in a collaborative and supportive team.
Ready to take the next step in your career Apply now!
Required Experience:
Senior Manager
Employment Type
Full-Time
