Job Description: We are looking for someone with a robust background in data engineering and ETL (Extract Transform Load). The candidate should have hands-on experience with Spark Python and SQL. Proficiency in building data pipelines using Azure is highly desirable. Additionally knowledge and experience with Power BI and Fabric including holding Azure certifications are considered a plus.
Core Job Responsibilities
- Responsible for working with product team to design and implement data solutions. This includes responsibility for the method and processes used to translate business needs into functional and technical specifications.
- Design develop and maintain robust data sets used for reporting and AI model serving.
- Responsible for building new Data Products in Azure as well as optimizing data flow and integrations with external customers.
- Write efficient and scalable Spark code creating high quality datasets that will be used for customer interaction and reporting.
- Participate in code reviews and contribute to best practices for data engineering.
- Troubleshoot and resolve data-related issues and provide support for escalated technical problems.
- Collaborate with cross-functional teams to understand data requirements and deliver solutions.
- Work with product owners and business users to understand and translate user stories and/or requirements into data sets analytics reports and machine learning pipelines.
Relevant Work Experience
- 5 years of experience in data engineering.
- Strong experience in Spark and ETL processes.
- Advanced SQL and Python knowledge and experience working with relational databases for data querying and retrieval.
- Knowledge and experience of building data lake and enterprise analytics solutions with Microsoft Azure including Azure Data Factory Azure SQL Database and other Azure services.
- Bachelors degree in computer science Information Technology or a related field.
Additional skills that are a plus but not required
- Experience with SAP HANA data
- Knowledge and experience with other cloud platforms like AWS and Google.
- Azure certifications (e.g. Microsoft Certified: Azure Data Engineer Associate) are highly desirable.
- Experience with Power BI / Tableau for reporting and data visualization.
Required Experience:
Senior IC