Data Engineer Databricks III

Hyderabad - India

Monthly Salary: Not Disclosed

Posted on: 5 hours ago

Vacancies: 1 Vacancy

Job Summary

Databricks Developer

We are looking for a skilled DatabricksDeveloper to design develop and
optimize big data solutions using ApacheSpark and Databricks on cloud
platforms like Azure AWS or GCP. The idealcandidate will have strong
experience in data engineering ETL pipelines andperformance
optimization.

Key Responsibilities:

Design develop and optimize ETL pipelinesin Databricks.

Build data ingestion transformation andprocessing workflows using
Apache Spark.

Work with structured and unstructured datafrom various sources.

Optimize Spark jobs for performance costefficiency and scalability.

Implement CI/CD for Databricks workflowsand manage version control.

Work closely with Data Engineers DataScientists and Business Analysts
to support analytics and reporting.

Implement security best practices andmanage access controls in
Databricks.

Troubleshoot performance and scalabilityissues in big data processing.

Required Skills:

Strong hands-on experience with Databricksand Apache Spark
(PySpark/Scala/Java).

Proficiency in SQL for data analysis andtransformation.

Experience in cloud platforms (Azure AWSor GCP) especially Azure Data
Lake AWS S3 or GCP BigQuery.

Hands-on experience with ETL tools DataPipelines and Data Warehousing.

Strong knowledge of CI/CD Git and DevOpstools for data workflows.

Knowledge of Delta Lake Lakehousearchitecture and performance tuning.

Experience in handling large-scale datasetsand distributed computing.

Preferred Skills:

Experience with Databricks MLflow andmachine learning pipelines.

Knowledge of Kafka Airflow or Snowflake.

Understanding of data governancecompliance and security policies.

Education & Experience:

Bachelors or Masters degree in ComputerScience Data Engineering or a
related field.

experience in Databricks development