AWS Databricks Platform Administrator

Job description

AWS Databricks Platform Administrator

Location Raritan NJ Onsite

We are looking for an experienced AWS Databricks Platform Administrator with 5 plus years of experience in cloud data platform administration with strong expertise in Databricks on AWS The role is responsible for managing securing automating and optimizing the Databricks platform to support enterprisescale data engineering analytics and governance needs The ideal candidate should have handson experience in AWS infrastructure IAM Unity Catalog cost optimization platform monitoring and production support

Key Responsibilities

Administer and manage Databricks workspaces in AWS environments

Configure and maintain secure access to data stored in Amazon S3 using IAM roles instance profiles and trust policies

Implement and manage Unity Catalog including catalogs schemas tables storage credentials and external locations

Define and enforce cluster policies access controls and workspace standards to ensure security and compliance

Monitor cluster performance usage trends and platform health and drive cost optimization through autoscaling tagging and policy controls

Automate platform provisioning and configuration using Infrastructure as Code tools such as Terraform

Troubleshoot production issues related to clusters permissions storage access networking and job failures

Collaborate with data engineering security cloud infrastructure and support teams to ensure platform reliability and governance

Support release management upgrades new feature enablement and platform best practices adoption

Create and maintain operational documentation SOPs and runbooks for platform administration

Required Skills and Qualifications

57 years of overall IT experience with strong handson experience in Databricks administration on AWS

Good understanding of AWS services including IAM S3 EC2 VPC CloudWatch and related security controls

Handson experience with Unity Catalog and enterprise data governance practices

Strong knowledge of Databricks cluster management workspace administration job monitoring and access management

Experience in Infrastructure as Code using Terraform and scripting using Python Shell or Bash

Knowledge of CICD release management and environment migration processes for platform changes

Experience in monitoring incident management root cause analysis and production support

Understanding of cost management and optimization techniques for Databricks and AWS resources

Strong communication and stakeholder management skills

Preferred Qualifications

Experience with Databricks E2 architecture and multiworkspace administration

Exposure to chargebackshowback tagging strategies and usage reporting

Familiarity with Databricks SQL dashboards and platform observability practices

AWS certifications such as AWS Certified Solutions Architect SysOps Administrator or Security Specialty

Experience working in enterprise data platform or analytics transformation programs

Skills

Mandatory Skills : AWS Cloud Architecture Databricks
Good to Have Skills : Aws Step Functions