Senior Data Engineer Databricks
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
Job Summary
As a Data Engineer you will play a pivotal role in designing building and optimizing our data pipelines and architectures. You will collaborate with cross-functional teams to ensure that our data infrastructure is robust scalable and aligned with our business goals. Your experience with fabrics and Databricks will be essential in driving our data strategy forward. Previous experinece in managing large set of data in P&C insurance is a must have for this role.
Heres What Youll Be Doing
Data Architecture & Pipeline Development:
- Design develop and maintain scalable data pipelines using Databricks.
- Implement and manage ETL processes to ensure high data quality and reliability.
- Optimize data storage solutions for performance and cost-efficiency.
- Coordinating with the Offshore team is a must have on this role.
Collaboration & Strategy:
- Work closely with data scientists analysts and other stakeholders to understand data requirements and deliver actionable insights.
- Contribute to the development of data strategies that align with the companys business objectives especially within the fabrics domain.
Data Management & Governance:
- Ensure data security privacy and governance across all data platforms.
- Implement best practices for data management including documentation testing and version control.
- Stay current with the latest trends and technologies in data engineering particularly within Databricks and the fabrics industry.
Heres What You Need
- 5 years of experience in data engineering or a related role.
- Extensive experience with Databricks including Databricks Delta Live Tables Databricks Asset Bundles.
- Proven experience in the MS fabrics with a strong understanding of industry-specific data challenges and opportunities.
- Hands-on experience with cloud platforms such as AWS Azure or GCP.
- Proficient in programming languages such as Python SQL.
- Strong SQL skills and experience with relational databases (e.g. MySQL PostgreSQL).
- Proficient with data visualization tools (e.g. Power BI Tableau)
- Proven experience in Dimensional Modelling including star schema basic notion
- Excellent problem-solving and analytical skills.
- Strong communication skills with the ability to translate complex technical concepts to non-technical stakeholders.
- Ability to work independently and as part of a team in a fast-paced environment.
- Informatica Power Center is nice to have!
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
We see opportunity in technology. In domains such as cloud, AI, mainframe modernisation, DLT and IoT, we blend established practice with new thinking to help our clients stay ahead.
