Sauvik Chatterjee

• Led multiple teams on critical projects - a. Migration for Amazon warehouse/associate performance data for all EU SCs - providing metrics like End of Shift, End of Day, TPH, b. Reduce wasted trucks by 2.3% through optimized truck scheduling forecast.
• Implemented GDPR compliance framework for extensive data discovery of PI data across data lakes and clusters, and providing hard/soft deletes on request, while setting up fine-grained access control, Data archival strategy, DQ & Usage metrics QuickSight dashboard.
• Architected Single Source Of Truth petabyte-scale Delta Lake (15 min Ingestion SLA) with Apache Hudi, onboarding batch & streaming datasets, handling delta & deletes, setting up Data discovery with LLMs, ML Feature Store & Data Lineages.
• Drove intiatives to migrate legacy infrastructure into Native AWS across org, built pipelines with Glue, Airflow, Containers, Dynamo DB, SageMaker, SQS, SNS, Cloudwatch dashboards for monitoring & alarms with Gitlab & CDK Infrastructure as Code, Unblocked Redshift clusters by implementing WLM & optimization, reducing execution time by 45%, held design reviews & mentoring sessions.

<p>Led multiple teams on critical projects for migration of Amazon warehouse/associate performance data for all EU SCs, providing metrics like End of Shift, End of Day, and TPH.<br>
Reduced wasted trucks by 2.3% through optimized truck scheduling forecast.<br>
Implemented GDPR compliance framework for extensive data discovery of PI data across data lakes and clusters.<br>
Provided hard/soft deletes on request.<br>
Set up fine-grained access control.<br>
Set up data archival strategy.<br>
Set up DQ & Usage metrics QuickSight dashboard.<br>
Architected Single Source Of Truth petabyte-scale Delta Lake with 15 min Ingestion SLA using Apache Hudi.<br>
Onboarded batch and streaming datasets.<br>
Handled delta and deletes.<br>
Set up data discovery with LLMs.<br>
Set up ML Feature Store.<br>
Set up data lineages.<br>
Drove initiatives to migrate legacy infrastructure into Native AWS across org.<br>
Built pipelines with Glue, Airflow, Containers, Dynamo DB, SageMaker, SQS, SNS, and Cloudwatch dashboards for monitoring and alarms.<br>
Used Gitlab and CDK Infrastructure as Code.<br>
Unblocked Redshift clusters by implementing WLM and optimization.<br>
Reduced execution time by 45%.<br>
Held design reviews and mentoring sessions.</p>

<p>Led a team of financial investors, data scientists and engineers to nowcast an important investment metric, Manufacturing PMI, 1 month before publication.<br>
Performed data extraction, cleansing, and EDA from webscraping financial news articles.<br>
Built NLP models like Financial BERT and LLMs for sentiment analysis and time series.<br>
Built LSTM for forecasting.<br>
Used dbt, Snowflake, and GCP.</p>

<p>Integrated data from HR and Operations Data lakes to create data products for Pre and Post-Covid Trend Analysis and Attendance Prediction models for worldwide MS offices.<br>
Validated KPIs like No Shows, Unique Badge Swipes, and Peak Average Attendance metrics by Leadership and Geography.<br>
Created Hybrid Workplace model global dashboard with a target of 50% Employee flexibility for Strategy and Leadership.<br>
Implemented with IoT data using Databricks Spark, ADF pipelines, Power BI, and CI/CD on Azure DevOps.<br>
Spearheaded ML in Sustainability initiative by ingesting and transforming HVAC IoT data, Badge Swipes, and Weather data.<br>
Improved accuracy metric in Smart Start and Stop time prediction by 4% with a target reduction of Carbon footprint by 30%.<br>
Implemented SARIMA and GBM models on Azure ML Studio deployed with Kubernetes, Synapse, and Databricks Spark and Delta lake.<br>
Led migration efforts for 4 teams of all Scala and Python pipelines and orchestration from Databricks to Synapse as Org PoC.<br>
Served as Team Lead and Scrum Master in a 6 member team.<br>
Executed SDLC.<br>
Ensured modular coding.<br>
Ensured Flake8 linting.<br>
Ensured testing frameworks.<br>
Worked in Agile framework.<br>
Collaborated with PO in writing and story-pointing User Stories on Jira.<br>
Estimated efforts and mitigated risks.</p>

<p>Provided consulting exposure to multiple US, Europe and Australia clients.<br>
Created advanced analytical solutions on RDBMS using ETL technologies like SSIS, ODI, Informatica, SAP BODS.<br>
Worked with NoSQL technologies such as Cosmos DB, Dynamo DB, Mongo DB, and Cassandra DB.<br>
Worked with AWS and Azure.<br>
Worked on multiple architectures, design patterns, batch and stream processing, file formats, and performance optimization.<br>
Built from scratch an in-house multitenant Enterprise Data Hub application for Goods & Services Tax using Kafka and Kinesis streaming.<br>
Developed AWS pipelines using On Demand and Auto scaled EMR, Step Functions, Redshift, Elasticsearch, and Kibana for real time ingestion dashboard.<br>
Performed Scala Spark tuning and optimization.<br>
Built monitoring solutions to terminate lingering EMRs.<br>
Received PwC Award.<br>
Migrated On-prem Big Data Applications and Databases architecture for a multinational bank into tiered Data Lake on Azure Datawarehouse and Cosmos DB.<br>
Implemented FIFO queuing mechanism for requests on Snowflake data loader.<br>
Forecasted predictive maintenance of Transformers for an Energy firm through Health Index dashboard with Alteryx.<br>
Ran ML models with Spark on Azure HDInsight with Apache LIVY.<br>
Used IOT sensor data from Cassandra.</p>

<p>Served as onshore Team lead of 5 Analysts from Amsterdam.<br>
Conducted regular meetings with Data Stewards and stakeholders at CxO level.<br>
Analyzed regions and modalities with 80% share loss, 3 year forecasts, and tracked KPIs including Market Pulse, Share Loss, and V Pts.<br>
Implemented centralized analytical capabilities by integrating data from 33 Operating Concerns.<br>
Performed dimensional modelling of UDM (Universal Data Model).<br>
Built Data Marts.<br>
Built Power BI dashboards and ETL pipelines with Azure Databricks, ADF, and DevOps.<br>
Led migration efforts from legacy ERP and CRM to SAP S/4 Hana Central Finance and Salesforce for a finance client in UK.</p>