POSITION SUMMARY

The Internal Audit Data Analytics team is seeking an experienced Databricks Data Engineer Lead to own the technical direction and delivery of Internal Audits Databricks-based analytics environment with strong experience integrating data from SAP ECC or SAP S/4HANA .This role leads the design and implementation of scalable governed data pipelines and lakehouse solutions that enable stand-alone audits continuous auditing and enterprise risk monitoring across the organization.

Reporting to the Data Analytics Sr. Manager Internal Audit the Data Engineer Lead is accountable for ensuring reliable ingestion data quality audit-ready lineage and operational excellence of Internal Audit datasets and pipelines. The role partners with auditors analysts IT teams and business stakeholders to deliver governed reusable datasets that reduce manual effort and improve audit coverage and timeliness.

This is a hands-on engineering leadership role requiring deep expertise in Databricks Spark Delta Lake and cloud data engineering plus the ability to lead delivery planning architecture decisions design reviews and mentoring across the team.

PRIMARY DUTIES AND RESPONSIBILITIES

Technical Leadership Architecture & Roadmap Ownership

• Own and evolve the data engineering roadmap for Internal Audits Databricks lakehouse including standards for ingestion patterns data modeling governance and operational reliability.

• Build and manage scalable ETL/ELT pipelines integrating data from SAP ECC or SAP S/4HANA

• Lead architecture and design reviews for new pipelines and enhancements; ensure alignment to engineering standards maintainability cost efficiency and audit requirements.

Delivery Leadership & Project / Program Management

• Lead end-to-end delivery for priority initiatives by translating business/audit needs into an executable delivery plan (scope milestones dependencies and resourcing).

• Coordinate execution across offshore resources and onshore teams to manage work intake backlog and prioritization to ensure delivery of reusable certified datasets and reliable pipelines supporting audits and risk monitoring.

• Proactively identify delivery risks (technical data availability access constraints performance upstream system changes) and drive mitigation plans.

Data Engineering & Platform Build

• Design build and maintain large-scale fault-tolerant pipelines using Python/PySpark Spark SQL Databricks Delta Lake and orchestration tools (Databricks Jobs ADF Airflow).

• Lead implementation of ingestion patterns across files databases APIs including streaming pipelines (Structured Streaming).

• Own implementation of CDC incremental loads and full refresh patterns; manage schema evolution and reconciliation for high-confidence audit datasets.

• Own curated lakehouse modeling (bronze/silver/gold) and publishing datasets for BI/analytics consumption.

• Drive performance and cost optimization for production workloads (partitioning Z-ORDER file sizing caching strategies cluster policies job tuning).

Data Quality Monitoring Governance & Auditability

• Establish and enforce data quality standards and implement automated checks anomaly detection routines monitoring and alerting aligned to SLAs.

• Establish repeatable processes for lineage documentation validation reconciliation and test coverage; ensure pipeline versioning and evidence is maintained for auditability.

• Implement scalable frameworks for metadata management and schema validation to support reliable and governed operations.

Audit Collaboration & Stakeholder Leadership

• Serve as the lead SME for Internal Audit on data engineering capabilities data availability Serve as the primary SME for data engineering within Internal Auditguiding stakeholders on data availability structures pipeline behavior and limitations.

• Partner with auditors/analysts to define requirements and deliver reusable datasets that accelerate audit execution and continuous monitoring.

• Collaborate with IT and business data owners to resolve access upstream data issues and changes that affect pipeline stability and audit outcomes.

Mentorship Standards & Team Enablement

• Establish consistency in engineering practices: code review standards documentation requirements development patterns testing discipline and CI/CD expectations.

• Mentor and coach engineers and analysts on Databricks engineering practices performance tuning and production readiness expectations.

• Contribute to hiring/interview loops and onboarding by defining technical screens building onboarding guides and promoting best practices across the team

.

EDUCATION AND EDUCATIONAL REQUIREMENTS

• Bachelors or masters degree in computer science Data Engineering Information Systems Analytics or related discipline (or equivalent work experience).

REQUIRED EXPERIENCE

• 8 years of relevant experience (data engineering analytics engineering platform engineering) with 5 years hands-on engineering in Databricks / Spark production environments & a team handling experience of 3 years.

• Handson experience with SAP ECC or SAP S/4HANA data extraction and processing.

• Demonstrated ownership of end-to-end data pipelines and lakehouse implementations (ingestion transformation/modeling publishing monitoring/operations).

• Proven experience acting as a technical lead including architecture decisions design reviews mentoring and delivery leadership across multiple initiatives.

REQUIRED TECHNICAL SKILLS

• Deep proficiency in Python PySpark/Spark and SQL including performance tuning practices and understanding of Spark execution concepts.

• Strong experience implementing Delta Lake patterns: table design upserts/merge logic schema enforcement/evolution incremental processing and optimization strategies.

• Experience with orchestration tools and production scheduling (e.g. Azure Data Factory Airflow Databricks Jobs).

• Experience with data quality monitoring and alerting patterns that support reliable SLA-driven operations.

• Strong engineering discipline: version control CI/CD practices and documentation/runbook creation.

PREFERRED EXPERIENCE / NICE TO HAVE

• Azure data ecosystem experience (e.g. ADLS ADF key enterprise patterns) and experience operating governed platforms in regulated environments.

• Experience building semantic/consumption layers and enabling BI/analytics usage patterns.

• Experience working effectively with US-based onshore stakeholders and offshore delivery teams.

PREFERRED CERTIFICATIONS

• Databricks Data Engineer (Associate / Professional).

• Microsoft Azure Data Engineer (or equivalent cloud data engineering certification).

CORE BEHAVIORAL COMPETENCIES

• Ownership & accountability; ability to make decisions and drive outcomes.

• Strong communication (technical and non-technical) stakeholder influence prioritization and execution under ambiguity.

• Coaching and mentorship mindset; raises the engineering bar through standards and examples.

What Cencora offers

Benefit offerings outside the US may vary by country and will be aligned to local market practice. The eligibility and effective date may differ for some benefits and for team members covered under collective bargaining agreements.

Affiliated Companies

Equal Employment Opportunity

Cencora is committed to providing equal employment opportunity without regard to race color religion sex sexual orientation gender identity genetic information national origin age disability veteran status or membership in any other class protected by federal state or local law.

The companys continued success depends on the full and effective utilization of qualified individuals. Therefore harassment is prohibited and all matters related to recruiting training compensation benefits promotions and transfers comply with equal opportunity principles and are non-discriminatory.

Cencora is committed to providing reasonable accommodations to individuals with disabilities during the employment process which are consistent with legal requirements. If you wish to request an accommodation while seeking employment please call 888.692.2272 or email . We will make accommodation determinations on a request-by-request basis. Messages and emails regarding anything other than accommodations requests will not be returned