Sr. Databricks Lead

Role: Sr. Databricks Lead

Location: Dallas TX (Onsite)

Role Descriptions: Sr. Databricks LeadJob role and Responsibilities A Sr. Databricks Developer designs builds and optimizes big data ETLELT pipelines using PySpark SQL and Delta Lake within cloud environments (Azure AWS or GCP). Key ResponsibilitiesKey responsibilities include creating scalable data solutions optimizing Spark jobs and managing data workflows to support analytics. Required skills include proficiency in PythonSQL Spark and experience with data warehousing concepts. Pipeline Development Design develop and maintain robust scalable ETLELT data pipelines using Databricks and Processing Work with structured and unstructured data utilizing Delta Lake for efficient storage and Perform performance tuning and optimization on Databricks clusters and Spark jobs for Use tools like Azure Data Factory (ADF) or Airflow for workflow orchestration and Integrate Databricks with cloud data sources (Azure Data Lake Gen2 S3) and BI Quality Security Implement data quality checks validation and security protocols. Required Skills QualificationsTechnical Skills Expert-level knowledge of PySpark Apache Spark and Platforms Hands-on experience with Azure Databricks (most common) AWS Databricks or Engineering Strong understanding of Data Modeling ETLELT patterns and Data Warehousing Languages Python Azure Data Factory (ADF) Delta Lake GitAzure 6-8 years of experience in data engineering or software development.
Essential Skills: Sr. Databricks LeadJob role and Responsibilities A Sr. Databricks Developer designs builds and optimizes big data ETLELT pipelines using PySpark SQL and Delta Lake within cloud environments (Azure AWS or GCP). Key ResponsibilitiesKey responsibilities include creating scalable data solutions optimizing Spark jobs and managing data workflows to support analytics. Required skills include proficiency in PythonSQL Spark and experience with data warehousing concepts. Pipeline Development Design develop and maintain robust scalable ETLELT data pipelines using Databricks and Processing Work with structured and unstructured data utilizing Delta Lake for efficient storage and Perform performance tuning and optimization on Databricks clusters and Spark jobs for Use tools like Azure Data Factory (ADF) or Airflow for workflow orchestration and Integrate Databricks with cloud data sources (Azure Data Lake Gen2 S3) and BI Quality Security Implement data quality checks validation and security protocols. Required Skills QualificationsTechnical Skills Expert-level knowledge of PySpark Apache Spark and Platforms Hands-on experience with Azure Databricks (most common) AWS Databricks or Engineering Strong understanding of Data Modeling ETLELT patterns and Data Warehousing Languages Python Azure Data Factory (ADF) Delta Lake GitAzure 6-8 years of experience in data engineering or software development.
Desirable Skills:
Keyword:
Skills: Digital : Databricks
Experience Required: 8-10