Snowflake DBT(Neha)

Job Summary

We are seeking a highly experienced Data Engineer (6-12 years) with expertise in Snowflake DBT Apache Airflow and StreamSets and strong hands-on experience in designing enterprise-grade ETL/ELT data migration and multi-source ingestion frameworks within the Life Sciences domain.

Key Responsibilities

1. Snowflake Architecture & Enterprise Data Platform Design

Lead architecture and implementation of scalable Snowflake data platforms:

o Multi-layered architecture (Landing Raw Staging Curated Data Marts)

Develop secure cross-account data sharing strategies.

Implement:

o Snowpipe for automated ingestion

o Streams & Tasks for CDC-based incremental processing

o Time Travel & Zero-copy cloning for environment management

Implement data masking row-level security and RBAC frameworks.

Optimize storage partitioning (micro-partition pruning) and query performance.

2. Data Migration & Modernization

Participate in end-to-end data migration initiatives including:

o Legacy data warehouse (Teradata Oracle SQL Server Netezza) to Snowflake

o On-prem to cloud modernization programs

o Source system analysis and profiling

o Data quality assessment and remediation planning

o Schema conversion and transformation mapping

o Incremental migration strategies

o Parallel-run validation strategies

Perform reconciliation and data validation between legacy and target systems.

Develop automated validation scripts using SQL and DBT tests.

Support cutover planning and production readiness.

3. Data Ingestion & Multi-Source Integration

Design and implement ingestion frameworks for structured semi-structured and unstructured data from multiple enterprise systems:

Structured Sources

Oracle SQL Server SAP PostgreSQL

Clinical systems (EDC CDMS CTMS)

Regulatory systems (RIM)

Commercial systems (CRM ERP)

Semi-Structured Sources

JSON XML Avro files

API responses

External vendor feeds

4. DBT Enterprise Transformation Framework

Design/ Develop DBT transformation layers:

o Staging models

o Intermediate models

o Data marts

Implement:

o Incremental models

o Snapshot strategies for historical tracking

o Surrogate key management

Develop custom macros and reusable transformation components.

Optimize DBT models specifically for Snowflake compute efficiency.

Required Qualifications

6-12 years of experience in Data Engineering and Enterprise Data Platforms.

4 6 years hands-on Snowflake implementation experience.

Strong experience in:

o Large-scale data migration programs

o Multi-source data ingestion frameworks

o DBT advanced transformation design

o Apache Airflow orchestration

o StreamSets ingestion pipelines

Advanced SQL expertise.

Experience in Life Sciences domain projects.

Cloud platform experience (AWS/Azure/GCP).