<p>Architected and led an enterprise data lake on Databricks (Unity Catalog, Delta Lake) and Fivetran, unifying 3 source types — Salesforce APIs, S3 JSON feeds, and regional databases — into a governed platform processing millions of records daily.</p><p> Engineered a metadata-driven ETL/ELT framework in collaboration with the Analytics team, supporting full and incremental loads, dynamic schema evolution, and centralized data quality checks.</p><p> Established a 3-layer Medallion Architecture (Bronze, Silver, Gold), creating reusable data products that simplified consumption for downstream analytical teams and accelerated time-to-insight.</p><p> Enforced Column-Level Security (CLS) and Row-Level Security (RLS) with Databricks SQL, aligned to the business hierarchy, securing PII across the data lake for GDPR compliance via encryption, masking, and comprehensive lineage documentation.</p><p> Developed a 360-degree partner profiling system combining customer segmentation and trend analysis, with a centralized metrics layer and automated nudges and personalized recommendations, enabling faster, data-driven decision-making for sales teams and driving increased sales and partner growth.</p><p> Captured change data (inserts, updates, deletes) for hundreds of thousands of daily events with AWS DMS CDC, writing Parquet files with operation type, timestamp, and source key to Amazon S3.</p><p> Built an idempotent Databricks ingestion job, increasing pipeline reliability to 99.9% and enabling safe, automated re-runs for hundreds of thousands of daily records.</p><p> Landed raw CDC events in an append-only Bronze Delta table to preserve full change history.</p><p> Merged Bronze into a deduplicated Silver Delta table via MERGE INTO, keeping the latest event per primary key.</p><p> Aggregated Silver into Gold tables and views for dashboards and reporting.</p><p> Led code reviews and mentored junior developers, improving team code quality and reducing operational costs through reusable pipeline patterns and best-practice documentation.</p>
