Databricks SME

Databricks SME

18 months

Washington D.C. 20224

Key Responsibilities

• Serve as the Databricks SME and guide technical decisions for workspace architecture design configuration and optimization.
• Lead governance processes within Databricks environments including access controls data lineage and workspace policies.
• Optimize Databricks cluster and job performance; identify and resolve bottlenecks in data pipelines.
• Design and implement secure integration patterns between Databricks and AWS services (e.g. S3 IAM Glue Redshift).
• Perform specialized operations including Delta Lake management Unity Catalog administration and MLflow tracking.
• Collaborate with government technical leads on architecture reviews and platform enhancement initiatives.
• Develop and maintain technical documentation runbooks and best practice guidelines.
• Support platform security requirements and ensure compliance with IRS/Treasury policies.
• Mentor mid-level engineers and contribute to team knowledge sharing.
• Participate in code reviews design discussions and agile ceremonies.

Required Qualifications & Skills

• 8 years of hands-on experience in cloud technology environments.
• 3 5 years of dedicated Databricks experience including workspace administration and optimization.
• Proven expertise as a Databricks SME with Unity Catalog Delta Lake and cluster management.
• Strong proficiency with AWS services including S3 IAM Glue Redshift and VPC networking.
• Experience with data governance frameworks and security controls in cloud environments.
• Proficiency in Python Scala or SQL for data engineering tasks.
• Familiarity with federal IT compliance standards (FedRAMP FISMA) preferred.
• Knowledge of tax-processing or financial data management systems is highly preferred.
• Databricks Certified Data Engineer Professional certification preferred.
• AWS Certified Data Analytics or Solutions Architect certification preferred.