Senior Databricks Engineer

We are looking for a Senior Databricks Engineer with 7 years of experience in data engineering and strong hands-on expertise in Databricks and Apache Spark. The ideal candidate will design and build scalable data pipelines optimize large-scale data processing and work with cloud-based data platforms.

• Design develop and maintain scalable data pipelines using Databricks and Apache Spark.
  
• Develop and optimize PySpark/Spark jobs for large-scale data processing.
  
• Build reliable ETL/ELT pipelines using Databricks workflows Delta Lake and SQL.
  
• Implement data transformations data quality checks and performance optimization.
  
• Work with cloud data platforms such as AWS Azure or GCP.
  
• Optimize Spark jobs cluster configurations partitioning joins and data processing performance.
  
• Collaborate with data engineers data scientists analysts and business teams to deliver data solutions.
  
• Implement best practices for CI/CD version control monitoring and production support.
  
• Troubleshoot data pipeline and performance issues in production environments.
  

• 7 years of experience in Data Engineering.
  
• Strong hands-on experience with Databricks.
  
• Strong expertise in Apache Spark / PySpark.
  
• Experience with Delta Lake Databricks Workflows and Spark SQL.
  
• Strong proficiency in Python and SQL.
  
• Experience building and optimizing large-scale ETL/ELT pipelines.
  
• Experience with at least one cloud platform: AWS Azure or GCP.
  
• Good understanding of data warehousing data modeling and distributed data processing.
  
• Experience with Git and CI/CD practices.
  

• Experience with Unity Catalog and Databricks governance.
  
• Experience with Azure Data Factory AWS Glue or similar orchestration tools.
  
• Knowledge of streaming technologies such as Kafka or Spark Structured Streaming.
  
• Experience with Infrastructure as Code or cloud automation.
  
• Databricks certifications are a plus.