Job Title: Senior Data Engineer Healthcare Domain
Location: New York NY
Job Type: Contract
Job Description:
We are seeking an experienced Senior Data Engineer with a strong background in healthcare data systems and hands-on expertise in Python Apache Spark and SQL. This role involves building robust data pipelines transforming and integrating data from various sources and supporting scalable data solutions that drive insights and compliance in the healthcare domain.
Key Responsibilities:
- Design develop and maintain scalable ETL pipelines and data architectures using Spark Python and SQL.
- Integrate and process data from various healthcare sources including EHRs claims systems and HL7/FHIR interfaces.
- Build and optimize data models in cloud environments (AWS Azure or GCP) and support data lake/lakehouse platforms.
- Work closely with Data Scientists Analysts and business teams to ensure data accuracy consistency and compliance.
- Implement data validation quality checks and transformation logic across large datasets.
- Collaborate with cross-functional teams to ensure data privacy HIPAA compliance and audit readiness.
- Troubleshoot data pipeline issues perform root cause analysis and ensure high data availability and performance.
- Participate in Agile/Scrum ceremonies and support sprint planning and delivery.
Required Qualifications:
- 8 years of professional experience in Data Engineering or related roles.
- 5 years of hands-on experience with Python Apache Spark (PySpark) and advanced SQL.
- Strong experience working with healthcare data (e.g. EDI 837/835 HL7 FHIR claims EMR/EHR).
- Expertise in building data pipelines in cloud environments (AWS Glue EMR Redshift S3 Azure Data Lake etc..
- Experience working with large-scale structured and unstructured datasets.
- Solid understanding of data warehousing concepts data governance and privacy regulations (HIPAA).
- Proficient in Bash/Shell scripting version control (Git) and CI/CD tools.
- Familiarity with tools like Airflow Informatica or DBT for orchestration and transformation.
Preferred Skills:
- Experience with healthcare interoperability standards (HL7 v2 CDA FHIR).
- Knowledge of data cataloging and lineage tools.
- Exposure to data quality frameworks and observability platforms.
- Background in working in Agile/Scrum teams and cloud-native architecture.
Soft Skills:
- Excellent communication collaboration and stakeholder management skills.
- Strong problem-solving abilities and attention to detail.
- Self-driven proactive and able to work in a fast-paced environment.