Nagur Shaik

<p>Led architecture and implementation of cloud-native data pipelines using Python, AWS Glue, and Lambda, enabling sub-minute ingestion into S3.</p><p> Delivered high-quality, hands-on data engineering solutions in an Agile environment, participating in full SDLC and sprint planning.</p><p> Maintained analytics-ready Snowflake cortex data foundations and built transformed data models using dbt for enterprise reporting.</p><p> Architected automated dbt data models and transformation pipelines to convert raw S3 datasets into analytics-ready Snowflake tables.</p><p> Engineered end-to-end cloud data migration solutions in AWS, assessing source workloads, defining migration strategies, and coordinating secure movement of enterprise datasets into AWS cloud environments.</p><p> Developed PySpark notebooks for complex transformation logic and joined data across different zones, applied Delta Lake format for time travel and data versioning, ensuring efficient incremental loads.</p><p> Support analytics-ready data foundations that can be extended for future AI and Cortex-related use cases without claiming direct implementation.</p><p> Designed AWS Glue and Step Functions workflows with Python and Lambda to extract claims, provider, and eligibility data from SQL Server, Oracle, and Salesforce into AWS S3 data lakes.</p><p> Developed and orchestrated AWS Glue and Step Functions workflows with Python to extract data from enterprise source systems and prepare datasets for downstream analytics.</p><p> Built AWS Glue ETL workflows to ingest, transform, catalog, and process migrated data within AWS, supporting scalable cloud-native data processing requirements.</p><p> Performed advanced SQL analysis across source and target databases to identify data inconsistencies, validate transformation logic, and ensure migration accuracy.</p><p> Analyzed database schemas, tables, relationships, dependencies, data volumes, and business rules to determine migration scope and establish source-to-target mapping specifications.</p><p> Built data migration workflows to move historical claims and member data from on-prem Oracle into AWS S3 and Snowflake.</p><p> Implemented real-time streaming using AWS Kinesis + Lambda to feed eligibility updates into Snowflake and Power BI.</p><p> Migrated legacy IBM DataStage jobs and PL/SQL scripts into cloud-native AWS Glue and dbt workflows, ensuring operational parity and reduced maintenance overhead.</p><p> Implemented migration validation and reconciliation processes by comparing source and target record counts, aggregates, null patterns, key fields, and business-critical data attributes.</p><p> Optimized data movement and transformation workflows by analyzing pipeline performance, partitioning strategies, processing volumes, and database query execution patterns.</p>

<p></p><ul><li>Led the architecture and implementation of a cloud-native data platform leveraging Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2. The solution enabled near real-time ingestion and transformation of healthcare data including claims, member eligibility, and provider information.</li><li>Delivered high-quality, hands-on data engineering solutions in an Agile environment, participating in full SDLC and sprint planning</li><li>Maintained analytics-ready Snowflake cortex data foundations relevant to future Snowflake Cortex and GenAI use cases.</li><li>Engineered end-to-end Azure to AWS data migration solutions, assessing source workloads, defining migration strategies, and coordinating secure movement of enterprise datasets into AWS cloud environments.</li><li>Developed PySpark notebooks for complex transformation logic and joined data across different zones, applied Delta Lake format for time travel and data versioning, ensuring efficient incremental loads.</li><li>Support analytics-ready data foundations that can be extended for future AI and Cortex-related use cases without claiming direct implementation.</li><li>Designed Azure Data Factory (ADF) integration pipelines to pull claims, provider, and eligibility data from SQL Server, Oracle, and Salesforce into Azure Data Lake (ADLS Gen2).</li><li>Developed and orchestrated Azure Data Factory (ADF) pipelines to extract data from Azure-based source systems, apply transformation logic, and prepare datasets for downstream AWS migration.</li><li>Built AWS Glue ETL workflows to ingest, transform, catalog, and process migrated data within AWS, supporting scalable cloud-native data processing requirements.</li><li>Performed advanced SQL analysis across source and target databases to identify data inconsistencies, validate transformation logic, and ensure migration accuracy.</li><li>Analyzed database schemas, tables, relationships, dependencies, data volumes, and business rules to determine migration scope and establish source-to-target mapping specifications.</li><li>Built data migration workflows to move historical claims and member data from on-prem Oracle into ADLS and Snowflake.</li></ul><p></p>

<p>Architected Snowflake Cortex AI search, extraction, and Generative AI LLM workflows over enterprise documents, reducing manual extraction effort by 40% with structured outputs.</p><p> Studied semantic and keyword-based search approaches to improve information discovery across business documents.</p><p> Evaluated document ingestion, text preparation, chunking, indexing, and retrieval workflows for enterprise knowledge-search use cases.</p>

<p>Designed conceptual retrieval workflows to make unstructured business information accessible through natural language queries.</p><p> Developed conceptual designs for AI assistants that interpret business questions, select appropriate tools, and generate context-aware responses.</p><p> Designed data ingestion pipelines from SAP, Salesforce, and third-party APIs into AWS S3 using AWS Lambda and Python, applying PySpark transformation logic for downstream query optimization.</p><p> Designed conceptual workflows combining data engineering pipelines and GenAI capabilities to support business intelligence use cases.</p><p> Studied LLM integration patterns with cloud data platforms and enterprise data processing workflows.</p><p> Created data models and metadata repositories, collaborating with architects to optimize transformation and scalability.</p><p> Conducted statistical analysis and anomaly detection, enabling optimization, forecasting, and BI insights.</p><p> Built AWS Glue ingestion of IoT telemetry/SAP/Salesforce into S3; transformed with PySpark to curated Silver/Gold layers for device health analytics.</p><p> Migrated telemetry marts from Teradata to Snowflake on AWS; validated with SQL reconciliation suites.</p><p> Implemented reconciliation reporting tailored to telemetry: Event-count parity per device/time bucket; late-arrival and out-of-order event detection. Checksum/hyperloglog cardinality checks to catch duplicates/missed partitions. Range/semantic rules (timestamp windows, sensor thresholds); violations quarantined with rule IDs. Dimensional completeness: device/customer/support relationships validated and reported. Pipeline SLO dashboard: ingestion delay, p95 processing time, freshness SLA, anomaly flags.</p><p> Supported AWS Glue, AWS S3, EMR, and Spark SQL for data ingestion and processing.</p><p> Led data migration and legacy ETL development using IBM DataStage, Oracle PL/SQL, and AWS Glue to transition workloads to Snowflake and Redshift.</p><p> Led data migration from Oracle/Teradata to AWS S3 + Redshift/Snowflake; re-platformed legacy ETL using Python, AWS Glue, Lambda, and Step Functions with SCD2 for financial history.</p><p> Built AWS Glue integration for policy/claims/investments into AWS S3 staging; PySpark transformations to curated marts powering regulatory reporting.</p><p> Instituted finance-grade reconciliation:</p>