Azure Databricks & Snowflake Data Engineer

Company Name:WillwareTechnologies

Role: Azure Databricks & Snowflake Data Engineer

Experience:8 Years

Location: Mumbai Chennai Bangalore Hyderabad Pune

WorkMode: Onsite

We are looking for an experienced Azure Databricks & Snowflake Data Engineer to design develop and optimize scalable cloud-based data solutions. The ideal candidate should have strong hands-on experience in Azure Databricks Snowflake PySpark SQL ETL/ELT and data warehousing.

The role involves building modern data platforms developing high-performance data pipelines implementing data engineering solutions and supporting enterprise analytics and data initiatives.

• Design and develop scalable data pipelines and ETL/ELT solutions using Azure Databricks and Snowflake.
• Develop and optimize data processing using PySpark and Spark SQL.
• Build data solutions using Delta Lake and implement reliable batch and streaming pipelines.
• Develop and manage Databricks Workflows for automated data processing.
• Implement data governance and security using Unity Catalog.
• Design and develop Snowflake data warehouse solutions and optimize query performance.
• Work with Snowpipe Streams & Tasks Dynamic Tables Time Travel and Secure Data Sharing.
• Develop and maintain data pipelines using Azure Data Factory (ADF).
• Work with Azure Data Lake Storage Gen2 (ADLS Gen2) for large-scale data storage and processing.
• Develop data solutions using Azure Synapse Analytics and Azure Functions.
• Implement CI/CD and deployment processes using Azure DevOps.
• Perform data modeling and develop solutions using relational databases.
• Optimize data pipelines Spark jobs SQL queries and Snowflake workloads for performance and cost.
• Collaborate with data architects analysts developers and business teams to deliver enterprise data solutions.
• Ensure data quality reliability security and scalability across data platforms.

• 8-10 years of experience in Data Engineering / Cloud Data Engineering.
• Strong hands-on experience with Azure Databricks.
• Strong expertise in PySpark and Spark SQL.
• Hands-on experience with Delta Lake.
• Experience with Databricks Workflows and Unity Catalog.
• Strong hands-on experience with Snowflake.
• Good knowledge of Snowflake Architecture.
• Experience with Snowpipe Streams & Tasks Dynamic Tables Time Travel and Secure Data Sharing.
• Strong SQL and Python programming skills.
• Experience in ETL/ELT development and data warehousing.
• Hands-on experience with Azure Data Factory and ADLS Gen2.
• Experience with data modeling and relational databases.
• Strong understanding of cloud-based data engineering and big data processing.

• Experience with Azure Synapse Analytics.
• Experience with Azure Functions.
• Experience with Azure DevOps and CI/CD.
• Knowledge of Shell Scripting.
• Experience working with SQL Server Oracle or PostgreSQL.
• Experience with real-time/streaming data processing.
• Knowledge of data governance security and enterprise data platforms.

Required Skills:

Data EngineeringAzure Lake.