Data engineer - Databricks

Core Skills

Experience 10 years

• Design and implement data architecture leveraging technologies such as Databricks Unity Catalog Privacera and Collibra.
• Develop optimize and manage data pipelines for ETL processes using Databricks with a focus on data integrity and quality.
• Design and maintain data models and schemas incorporating Unity Catalog and Collibra data governance practices.
• Ensure data security and compliance with regulations using Databricks and Privaceras features.
• Establish a robust data governance strategy defining standards metadata management lineage and quality practices.
• Operationalize Machine Learning models in Batch and Real Time Data Pipelines leveraging relevant governance setups.
• Collaborate with crossfunctional teams including data scientists engineers and analysts to translate business requirements into scalable solutions.