Senior Data Engineer
Job Location:
Yearly Salary:
Posted on:
Vacancies:
Job Summary
The Lead Data Engineer will design build and operationalize scalable data solutions to support enterprise analytics and AI/ML initiatives. This role requires expert-level proficiency in Databricks Azure Fabric PySpark SQL and the Azure ecosystem with deep experience across data warehouses data lakes and real-time integration. The Lead Data Engineer will architect end-to-end pipelines using industry-standard tools drive automation and move solutions effectively into production. The incumbent will ensure compliance with data governance requirements (including GxP and HIPAA/GDPR) while building reusable integrated pipelines and analytical models that promote self-service analytics. This role provides technical leadership across the team mentors junior engineers and partners with business stakeholders to align data engineering with organizational objectives.
About the Role
. Data Architecture & Engineering
- Architect design and implement end-to-end data solutions using Azure Databricks PySpark Azure Data Factory and Azure SQL.
- Design build and maintain data pipelines from data sources through integration to consumption for specific use cases.
- Implement robust data modeling standards across bronze silver and gold layers in the data lake.
- Develop data models (conceptual logical and/or physical) as required.
- Optimize Spark and SQL workloads for performance scalability and cost efficiency.
- Manage metadata using data preparation integration and AI-enabled tools and techniques.
. Data Integration & Automation
- Drive automation in data integration; recommend and lead implementation of techniques to automate repeatable data preparation and integration tasks.
- Build API-based integrations (REST/JSON) and real-time ingestion frameworks.
- Automate data workflows using Azure DevOps pipelines and Git-based CI/CD practices.
- Implement parameterized reusable pipeline templates for ingestion and transformation.
- Develop automated unit regression and integration testing frameworks for data jobs.
. Analytics & Data Enablement
- Prepare and curate high-quality datasets for BI reporting and advanced analytics.
- Partner with analytics teams using Power BI Tableau or similar platforms to define semantic models and KPIs.
- Implement performance-optimized data models for self-service analytics.
- Will occasionally provide support to end users on the use of data visualization solutions.
Stakeholder Engagement & Leadership
- Lead technical design reviews mentor junior engineers and promote best practices.
- Assist cross-functional groups business analysts and stakeholders to gather define and refine data requirements.
- Collaborate with business and IT stakeholders to align data engineering with organizational objectives.
- Propose innovative data ingestion preparation and integration techniques to address stakeholder requirements.
- Contribute to architectural roadmaps and technology evaluations for the data platform.
- In collaboration with functional leaders identify inefficiencies and recommend improvements to the executive team.
About You
Job Experience & Education Requirements:
Bachelors degree in Computer Science Information Systems Engineering or related field (Masters preferred)
And
58 years of experience designing and developing enterprise-scale data solutions (data warehouses data lakes operational databases)
Other:
- Expert-level proficiency in Databricks Azure Fabric PySpark SQL and Azure DevOps.
- Proven experience with Azure Data Factory ADLS Gen2 and Azure SQL Server.
- Strong experience with Microsoft Azure data management architectures including Data Warehouse Data Lake and Data Catalogue and supporting processes such as Data Integration Governance and Metadata Management.
- Experience with Power BI required; Tableau or Looker a plus.
- Working knowledge of CI/CD automation version control (Git) and infrastructure as code (ARM Bicep or Terraform).
- Experience in life sciences or healthcare industries is a strong plus.
- Good understanding of GxP GDPR/HIPAA and applicable CFR/CTR/CTD regulations.
- Demonstrated success working with both IT and business stakeholders while integrating analytics and data science output into business processes and workflows.
- Must have excellent written and verbal communication skills.
- Proven ability to work independently and as part of a team and meet important deadlines.
- Statistical analysis skills are an asset.
Required Experience:
Senior IC
About Company
Accelerate your GI clinical trials with Alimentiv’s site network, clinical trial services, medical imaging, and biomarker discovery
