Job Description:
Databricks & Spark: Strong hands-on experience with Databricks Apache Spark and Delta Lake.
SQL Expertise: Advanced SQL skills for data transformation validation and performance tuning.
Lakehouse Patterns: Practical experience implementing Bronze/Silver/Gold data layers.
Cloud Platforms: Experience with at least one major cloud provider (Azure AWS or GCP).
Data Engineering Practices: Experience with batch ingestion incremental processing and structured data transformations.
Version Control & CI/CD: Familiarity with Git-based workflows and CI/CD pipelines.
Good-to-Have Skills
Streaming technologies (Kafka Event Hubs Kinesis Structured Streaming).
Databricks Unity Catalog usage (permissions schemas metadata).
Metadata-driven ingestion frameworks.
dbt or similar transformation tools.
Data quality or observability tooling.
Infrastructure as Code (Terraform or equivalent).
Exposure to ML/AI-adjacent data preparation or feature engineering.