Azure Databricks Data Engineer3 to 5 Years Gyansys
Azure Databricks Data Engineer3 to 5 Years Gyansys
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Description: Data Engineer (Azure Databricks & PySpark)
Position: Data Engineer
Experience: 3 to 5 years
Primary Skills: Azure Databricks PySpark SQL
Secondary Skills: ADF (Azure Data Factory)
Location: Bengaluru/Hyderabad
Mode of Work: Hybrid
Notice Period: 0 to 30 Days
Key Responsibilities:
ETL Development & Data Processing:
- Design develop and maintain ETL pipelines using Azure Databricks PySpark and ADF.
- Extract transform and load data from multiple sources into Azure Data Lake Storage.
- Optimize and finetune ETL workflows for performance and scalability.
SQL & Query Optimization:
- Write complex SQL queries joins subqueries and functions for data processing.
- Perform SQL tuning and query optimization for improved efficiency.
- Work with large datasets and ensure optimal data handling.
Azure Cloud Integration:
- Implement data solutions using Azure technologies (Databricks ADF ADLS).
- Work with batch and streaming data processing for realtime analytics (preferred).
- Ensure data security compliance and reliability in cloudbased systems.
Collaboration & ProblemSolving:
- Work closely with crossfunctional teams to support data analytics and business intelligence initiatives.
- Identify and resolve datarelated issues proactively.
- Provide insights and recommendations for improving ETL performance.
Required Skills & Qualifications:
3 to 5 years of experience in Azure Databricks PySpark and SQL.
Strong SQL skills complex joins functions procedures query tuning.
Handson experience in ADF (Azure Data Factory) and Azure Data Lake Storage.
Ability to handle large datasets efficiently.
Experience in performance optimization of ETL workflows.
Good understanding of batch and streaming data processing (preferred).
Strong analytical and problemsolving skills.
Ability to work independently and take ownership of tasks.
Employment Type
Full Time
