DataBricks Administrator
DataBricks Administrator
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
DataBricks Administrator
Onsite and local to Cincinnati.
Key Responsibilities:
Azure Databricks Administration & Management:
- Deploy configure and manage Azure Databricks workspaces in a scalable costefficient and secure manner.
- Administer clusters jobs notebooks and workflows ensuring high availability and performance.
- Monitor and optimize compute resource utilization and autoscaling strategies to improve cost efficiency.
- Manage Databricks Runtime versions libraries and dependencies across environments.
Security & Compliance:
- Implement and manage Unity Catalog for finegrained access control and data governance.
- Enforce RoleBased Access Control (RBAC) and integrate Databricks with Azure Active Directory (AAD).
- Ensure compliance with SOC 2 HIPAA GDPR and internal security standards.
- Set up audit logging monitoring and alerting for security and operational insights.
Performance Optimization & Troubleshooting:
- Tune Apache Spark workloads to improve query performance and resource efficiency.
- Analyze and troubleshoot performance bottlenecks in ETL and ML workloads.
- Optimize Delta Lake storage caching and indexing strategies for better query execution.
Automation & Infrastructure as Code (IaC):
- Automate Databricks workspace deployment using Terraform ARM Templates or Databricks REST API.
- Develop and maintain CI/CD pipelines for Databricks job deployment and configuration management.
- Implement monitoring solutions using Azure Monitor Prometheus or Grafana.
Collaboration & Integration:
- Work closely with data engineers data scientists and DevOps teams to support data pipelines and analytics workloads.
- Integrate Databricks with Azure Data Lake Azure Synapse Analytics and Snowflake.
- Provide technical guidance and best practices for efficient Spark job execution and cost optimization.
Required Skills & Experience:
- 5 years of experience in Azure Databricks administration and performance optimization.
- Expertise in Apache Spark PySpark SQL and Scala.
- Handson experience with Databricks Unity Catalog Delta Lake and MLflow.
- Strong knowledge of Azure cloud services (Azure Data Lake Azure Synapse Azure Key Vault etc.).
- Experience in Infrastructure as Code (Terraform Bicep or ARM Templates).
- Proficiency in CI/CD pipeline automation using Azure DevOps GitHub Actions or Jenkins.
- Strong understanding of network security identity management (AAD) and encryption best practices.
- Excellent problemsolving skills and ability to troubleshoot complex Databricks workloads.
- Strong communication and documentation skills.
Preferred Qualifications:
- Databricks Certified Associate or Professional certification.
- Experience with Azure Kubernetes Service (AKS) and serverless computing.
- Familiarity with Kafka Apache Airflow and eventdriven architectures.
- Knowledge of Python PowerShell or Bash scripting for automation.
Employment Type
Full Time
