Senior Data Engineer Databricks
Job Location:
Indianapolis, IN - USA
Monthly Salary:
Experience Required:
Posted:
Application Deadline:
Vacancies:
Job Summary
Job Title: Senior AWS Data Engineer
Location: Indianapolis IN
Role Overview
- Design develop and maintain data ingestion pipelines for real-world data (RWD) sources
- Build and optimize data models within a Databricks Lakehouse architecture
- Ensure data governance quality and compliance standards are met
- Optimize cost performance and resource utilization across AWS and Databricks
- Collaborate with business and technical stakeholders to translate requirements into scalable solutions
- Provide technical guidance coding standards and best practices
- Support knowledge sharing through documentation and training sessions
- Deliver status updates and progress reports to senior leadership
Required Qualifications
- Strong hands-on experience with AWS services (S3 Glue Lambda Redshift etc.)
- Proven experience with Databricks (Delta Lake notebooks workflows)
- Expertise in SQL and Python
- Experience building end-to-end ETL/ELT data pipelines
- Strong understanding of data modeling techniques
- Experience with data governance quality and compliance frameworks
- Ability to work independently and in a collaborative team environment
- Databricks Certification
- Experience building and managing CI/CD pipelines
- Experience with workflow orchestration tools
- Exposure to data observability and lineage tools
- Experience with monitoring logging and observability frameworks
Highly Preferred Skills
- Experience in Healthcare / Pharma domain
- Exposure to RWD (Real World Data) or HEOR analytics
- Familiarity with Lakehouse architecture / Medallion (Bronze-Silver-Gold) models
- Experience optimizing cost and performance in cloud environments
- Strong communication and stakeholder management skills
Required Skills:
Required Qualifications Strong hands-on experience with AWS services (S3 Glue Lambda Redshift etc.) Proven experience with Databricks (Delta Lake notebooks workflows) Expertise in SQL and Python Experience building end-to-end ETL/ELT data pipelines Strong understanding of data modeling techniques Experience with data governance quality and compliance frameworks Ability to work independently and in a collaborative team environment Databricks Certification Experience building and managing CI/CD pipelines Experience with workflow orchestration tools Exposure to data observability and lineage tools Experience with monitoring logging and observability frameworks Highly Preferred Skills Experience in Healthcare / Pharma domain Exposure to RWD (Real World Data) or HEOR analytics Familiarity with Lakehouse architecture / Medallion (Bronze-Silver-Gold) models Experience optimizing cost and performance in cloud environments Strong communication and stakeholder management skills
Required Education:
Bachelors Degree
