Lavanya Mohan

• As a senior data engineer, Gather and analyze the business requirements and then translate them to technical specifications.
• Created snowflake external tables on files (json, parquet and csv) stored in AWS S3 by using DBT external tables package
• Created snapshots, SCD Type-2 dimensions and Facts with DBT by following Kimbal dimensional modeling
• Worked with downstream stakeholders and created views and reporting tables for their use cases
• Well versed with DBT cloud, developed and scheduled orchestration of DBT models
• Well versed with different DBT materializations and tuned incremental models
• Created documentation on how to set up and use DBT core for the local dev environment.
• Maintained and upgraded DBT core from v1.0 to v1.5
• Worked on 2 Bigdata project use cases which includes following activities. Building a data pipeline to feed a Machine Learning model which will predict the product grade of abnormal/deviated materials as part of chip manufacturing process. This will reduce manual engineer intervention on the manufacturing floor and help in yield increase.
• Created several DBT macros for objects deletion, backups and other maintenance activities
• Trained team on best practices on how to use DBT tests, documentation and exposures. Integrated DBT elementary package with DBT project
• Well versed with snowflake query optimization, roles and administrative activities.
• Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, Spark SQL, and Data Lake Analytics. Data Ingestion to one or more Azure Services - (Data Lake, Storage, SQL, Azure DW) and data processing in Data bricks.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like SQL, Blob storage, SQL Data warehouse, write-back tool, and backward.
• Developed Spark applications using python
• Design, develop, and maintain scalable and robust data pipelines using Python and frameworks like Apache Airflow or Luigi. Integrate pipelines with AWS services such as S3, Redshift, or DynamoDB for data storage and processing.
• Implement ETL processes to extract data from various sources, transform it into a s

Gather and analyze the business requirements and then translate them to technical specifications.
Created snowflake external tables on files (json, parquet and csv) stored in AWS S3 by using DBT external tables package.
Created snapshots, SCD Type-2 dimensions and Facts with DBT by following Kimbal dimensional modeling.
Worked with downstream stakeholders and created views and reporting tables for their use cases.
Well versed with DBT cloud, developed and scheduled orchestration of DBT models.
Well versed with different DBT materializations and tuned incremental models.
Created documentation on how to set up and use DBT core for the local dev environment.
Maintained and upgraded DBT core from v1.0 to v1.5.
Worked on 2 Bigdata project use cases which includes following activities.
Building a data pipeline to feed a Machine Learning model which will predict the product grade of abnormal/deviated materials as part of chip manufacturing process.
This will reduce manual engineer intervention on the manufacturing floor and help in yield increase.
Created several DBT macros for objects deletion, backups and other maintenance activities.
Trained team on best practices on how to use DBT tests, documentation and exposures.
Integrated DBT elementary package with DBT project.
Well versed with snowflake query optimization, roles and administrative activities.
Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data.
Understand current Production state of application and determine the impact of new implementation on existing business processes.
Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, Spark SQL, and Data Lake Analytics.
Data Ingestion to one or more Azure Services - (Data Lake, Storage, SQL, Azure DW) and data processing in Data bricks.
Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like SQL, Blob storage, SQL Data warehouse, write-back tool, and backward.
Developed Spark applications using python.
Design, develop, and maintain scalable and robust data pipelines using Python and frameworks like Apache Airflow or Luigi.
Integrate pipelines with AWS services such as S3, Redshift, or DynamoDB for data storage and processing.
Implement ETL processes to extract data from various sources, transform it into a suitable format using Python libraries like Pandas or PySpark, and load it into data warehouses or data lakes on AWS.
Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into customer usage patterns.
Hands-on experience in developing SQL Scripts for automation purposes.
Responsible for estimating the cluster size, and monitoring and troubleshooting of the Spark data bricks cluster.
Implemented Automated Mails using ADF and Logic Apps.
Built end-to-end Data pipelines using Pyspark to process billions of records, performs several transformations based on business rules.
Design and implement data storage solutions using Azure services such as Azure SQL Database, and Azure Data Lake Storage Gen2.
Developed and maintained data pipelines using Azure Data Factory and Azure Databricks, Azure Synapse Analytics and worked on Parquet and JSON formats.
Perform data modeling and schema design for efficient data storage and retrieval.
Optimize data processing and storage for performance and cost efficiency.
Developed data models that streamlined data processing pipelines in the Azure environment, resulting in an increase of 30% in productivity.
Data modeling in Power BI to create a Star Schema with Facts and Dimension table.
Executing Unit testing on the data showing in dashboard visuals v/s raw data files.
Implementing row-level security on data along with an understanding of application security layer models.
Design and implement data pipelines using Python frameworks like Apache Airflow or Luigi on AWS infrastructure.
Build and deploy serverless applications using AWS Lambda, API Gateway, and DynamoDB with Python.
Develop scalable applications using Scala and Apache Spark for data processing and analytics on AWS EMR or Databricks.
Implement real-time streaming applications using Scala, Apache Kafka or AWS Kinesis.
Design and optimize data pipelines using Scala, Spark RDDs, DataFrames, and SQL on AWS S3, Redshift, or DynamoDB.
Deploy, configure, and manage Apache Spark clusters on AWS EMR or Databricks.
Develop and optimize Spark jobs for large-scale data processing and ETL on AWS S3 or HDFS.
Implement real-time analytics and streaming data pipelines using Spark Streaming on AWS Kinesis or Kafka.
Integrate data from multiple sources and formats, ensuring data quality, consistency, and reliability.
Utilize Python for data cleansing, normalization, and transformation tasks to prepare data for analytics and reporting.
Design and implement scalable Snowflake data architectures on AWS, including data warehouses, data lakes, and data marts.
Utilize Snowflake features like virtual warehouses and clustering to optimize query performance.
Develop and maintain ETL pipelines to extract, transform, and load data into Snowflake using AWS services such as AWS Glue, AWS Data Pipeline, or custom Python scripts.
Ensure data integrity and consistency across different data sources.
Monitor and optimize Snowflake performance on AWS by tuning SQL queries, configuring virtual warehouses, and optimizing data loading strategies.
Implement best practices for query optimization and resource utilization.

Streamlined and optimized ETL pipelines using Snowflake, reducing data processing time by 30% and enabling faster claims processing.
Worked on stringifying Json data and parsing Json string using functions to flatten the files using lateral flattening feature and performed complex SQL for data manipulation.
Used time travel queries in snowflake for data retention to access and query historical data at different points in time.
Created and configured snowflake warehouse strategy to move terabytes of data from Amazon S3 into Snowflake via PUT scripts.
Collaborated with business analysts to develop real-time dashboards, providing real-time insights into claims data, and facilitating proactive decision-making to improve customer experiences.
Facilitated productive cross-team communication to bridge the gap between data engineering and business needs, resulting in improved project alignment.
Used Jira and Agile methodology for managing tasks.
Analyze payroll and benefits trends, and Co-ordinate with Payroll and Human Resources to ensure accurate payroll reporting.
Periodic auditing of internal functions through assessment of processed data.
Excel in a team-oriented, collaborative environment while contributing to the creation, design, and implementation of value-add business strategies that affect current operating practices and company policy.
Ensure Payroll Processing for Jamaica is successfully completed on a timely basis- Track & Monitor Metrics for Quality and OTD for all these locations.
Timely processing of Statutory payments to Financial Institutions and Processing of all employees related Forms as per the defined SLAs.
Prepare and Publish MIS Reports for Jamaica on a timely basis with Quality data for review & analysis.
Exhibit strong communication and listening skills, focused on conveying new ideas to performance managers and understanding potential obstacles to implementation.
Lead the design and implementation of end-to-end data pipelines on Azure, resulting in a 40% improvement in data processing efficiency.
Hands on experience in Azure Data Factory (ADF), SQL Azure, Azure Data Storage.
Hands-on experience in Azure Analytics Services – Azure Data Lake Store (ADLS), Azure BLOB Storage, Azure SQL DB, Azure Data Factory (ADF) etc.
Retrieve data from the data warehouse structure and Perform query optimization and performance tuning.
Mentor team members in SQL programming.
Created packages in SSIS Designer using Control Flow Tasks and Data Flow Transformations to implement Business Rules.
Creating the Mappings using Data Flow Transformations such as the Sort, Derived Column, Conditional Split, SCD, Pivot and Lookup etc.
Using Control Flow, elements like Containers, Execute SQL Task, Execute Package Task, File System Task and Send Mail Task etc.
Scheduling and Monitoring the ETL packages for Daily load in SQL Server Job Agent.
Debugging and Validating the Data flows and Control Flows.
Be able to clearly articulate and present ideas and findings to a varied audience including the operations staff and senior management.
Define analytical approaches to problem-solving, reviews and vets with stakeholders.
Perform root cause an analysis and recommends holistic solutions leading to systemic and sustainable business and process improvement.
Implement machine learning models using Python libraries (e.g., scikit-learn, TensorFlow) on AWS SageMaker or EC2 instances.
Design and optimize data pipelines using Scala, Spark RDDs, DataFrames, and SQL on AWS S3, Redshift, or DynamoDB.
Fine-tune Scala applications for performance and scalability on AWS EC2 instances or serverless architectures.
Utilize Spark MLlib for scalable machine learning model training and inference on AWS infrastructure.
Integrate Spark with AWS services like S3, Redshift, DynamoDB, and RDS for data warehousing and analytics.
Administer and manage Aerospike or Solr databases on AWS EC2 or managed services like RDS or DocumentDB.
Optimize SQL queries for performance and efficiency on Aerospike or Solr databases.
Design and implement data models and schemas for Aerospike or Solr databases based on application requirements.
Configure and maintain indexes to support efficient data retrieval and search capabilities.
Implement HA/DR solutions for Aerospike or Solr databases on AWS using replication, backups, and failover strategies.
Utilize Python with Apache Spark or other distributed computing frameworks to process large volumes of data efficiently.
Implement batch processing and real-time data streaming solutions on AWS infrastructure.
Monitor data pipelines and job performance, identifying bottlenecks and optimizing processes for improved efficiency and reliability.
Implement logging, alerting, and automation using Python and AWS CloudWatch or other monitoring tools.
Implement data security measures and compliance standards (e.g., GDPR, HIPAA) within Snowflake on AWS.
Configure access controls, encryption, and auditing features to ensure data protection and regulatory compliance.
Establish data governance policies and practices for managing metadata, data lineage, and data cataloging within Snowflake on AWS.
Document data models, schemas, and data flows to ensure transparency and maintainability.

Analyze financial performance metrics and key performance indicators (KPIs) to identify trends, risks, and opportunities.
Prepare monthly, quarterly, and annual financial reports and presentations for management and stakeholders.
Assist in the development and implementation of financial processes, policies, and procedures to improve efficiency and effectiveness.
Collaborate with cross-functional teams to gather financial data, perform ad-hoc analysis, and support business initiatives.
Monitor and analyze industry trends, economic factors, and competitive benchmarks to inform financial forecasts and projections.
Support the annual budgeting process and periodic forecasting exercises, including revenue, expenses, and capital expenditures.
Provide financial support and analysis for strategic projects, acquisitions, and investment opportunities.
Ensure compliance with regulatory requirements and internal controls related to financial reporting and analysis.
Engineered and maintained data transformation pipelines, optimizing data for analysis, resulting in a 25% improvement in reporting efficiency, while preserving 99% data integrity.
Assured data availability, reliability, and security by managing and security by managing and maintaining databases, achieving 100% data uptime.
Orchestrated ETL processes, automating data workflows and enhancing data workflows and enhancing data ingestion, reducing processing time by40%.
Designed and developed comprehensive data pipelines using Azure Data Factory (ADF), facilitating seamless data migration from on-premises systems to Azure SQL Server and Azure Data Lake.
Built and managed complex ETL jobs using Azure Databricks, Pyspark, and Spark SQL, implementing robust error handling mechanisms to ensure data integrity and reliability.
Utilized Azure Data Factory for various data movement activities including copy, data flow, and control activities, and implemented performance tuning techniques to enhance efficiency in ADF and Azure Synapse Analytics.
Configured Azure Logic Apps to automate email notifications for end users and key stakeholders, improving operational workflows and communication.
Managed data storage and processing with Big Query and Hadoop, enhancing data accessibility and efficiency, and developed ETL pipelines using Teradata and Apache Spark.
Orchestrated data workflows with Apache Airflow, ensuring automated task management and error monitoring.
Leveraged Azure Key Vault to manage secrets and enhance security for linked services configurations.
Developed and managed Azure Stream Analytics jobs to process real-time data using Azure Event Hubs, and configured Azure Data Factory Triggers for scheduling pipelines and monitoring alerts.
Implemented Slowly Changing Dimensions (SCD-1 and SCD-2) methodologies in Databricks for effective data versioning and storage optimization using parquet formats.
Developed applications to consume data from Kafka producers and push messages to HDFS for further processing.
Employed Agile methodologies and utilized JIRA for managing project timelines and deliverables effectively.
Integrated Azure Data Lake and Blob storage with Databricks for data analysis and processing and configured new Event Hubs for event-driven data processing.
Streamlined ETL processes using Informatica PowerCenter, ensuring data consistency and reliability in data warehousing.
Utilized Azure Resource Manager (ARM) templates for consistent and reproducible deployments via Azure DevOps.
Developed logical and physical data models for Snowflake, aligning with necessary changes and data requirements.
Managed and queried databases using DBeaver, enhancing development, testing, and maintenance workflows.
Extensively worked with various Azure cloud services including Azure Blob Storage, Azure Functions, and Azure AD for comprehensive cloud solutions.
Created and maintained optimal data pipeline architecture in Microsoft Azure using Data Factory and Databricks, applying performance tuning techniques to optimize Spark jobs.

Contributed to the design of data models, supporting analytics and reporting, resulting in a 30% increase in data-driven decision-making.
Crafted comprehensive documentation for data sources and transformation processes, promoting knowledge sharing and data lineage transparency.
Provided expert technical support, resolving data-related issues with a 99% issue resolution rate.
Stayed at the forefront of industry best practices and emerging technologies enhancing, data processes by 20%.
Part of a Client-serving team and working with the multiple team members in getting the process and tasks completed efficiently.
Utilize various tools to create custom reports and update financial models used to budget and forecast project financials, KPIs, and profit and loss data.
Liaise extensively with various stakeholders while ensuring overall project tracking and planning along with reviewing the daily set of deliverables.
Conducted thorough analysis of business processes, identified pain points, and recommended process improvements that resulted in a 15% increase in operational efficiency.
Developed comprehensive business requirements documents, functional specifications, and user stories to guide the development and implementation of software solutions.
Conducted a comprehensive process review, reducing operational costs by 20% through improved efficiency and resource allocation.
Chosen by management to spearhead a newly formed team for a strategic client, responsible for leading the team's efforts and facilitating the seamless integration of new hires into the team.
Designed, built, and maintained efficient ETL pipelines using Azure Data Factory and Databricks to ensure seamless data integration and transformation.
Supported the development and maintenance of data warehousing solutions on Azure, including Azure SQL Data Warehouse (Synapse Analytics) and Azure Data Lake, ensuring data is properly stored and accessible.
Implemented data quality checks and governance practices to maintain data integrity, consistency, and security across all data pipelines.
Monitored and optimized data pipelines and storage solutions for enhanced performance and cost-efficiency, ensuring quick and reliable data access.
Worked closely with data scientists, analysts, and other stakeholders to understand data requirements, ensuring solutions met business needs and provided valuable insights.
Created and maintained comprehensive documentation for data pipelines, data models, and processes, ensuring clarity and continuity within the team.

Creating algorithms based on deep-dive statistical analysis & predictive data modeling to personalize customer interactions Analyzing customer habits & creating user-friendly reports using data manipulation to boost subscribers by 32%.
Deploying quantitative analysis & data mining/visualization to present data on user interaction with the product Identifying and analyzing the impact of marketing and product changes on customer acquisition & behavior, and retention.
Organizing huge data sets effectively via advanced querying, visualization, and analytics tools.
Performing data analysis & creating dashboards to boost flagship business initiatives across media relations, HR, and legal departments.
Playing a pivotal role in generating $8 million in ad revenue by preparing data sets & creating dashboards to boost sales.
Contributing to the formulation of strategies across multiple business units, distribution channels, and product lines.
Evaluating and formulating enhancement strategies to boost KPIs across all business units.
Training the recruits in data storage structures and data cleansing while managing & maintaining master data.
Conducted business requirements gathering, designed data flows, performed data quality analysis, and collaborated with the data warehouse architect to develop logical models, driving efficient data management.
Developed robust ETL processes using SSIS, extracting data from diverse sources, applying data transformations, and loading data into destination systems for efficient data integration and consolidation.
Analyzed database tables and wrote stored procedures and views in MS SQL Server and performed query optimization and performance tuning.
Designed and developed SSIS packages for streamlined data movement from staging to EDW environments, enhancing data integration efficiency within the organization.
Designed, Developed and Deployed servers in MS SQL Server environment using the SSIS Packages and created jobs for efficient running of the packages.
Developed and maintained a data store for reporting in SSRS, utilizing Global Variables, Expressions, and Functions to generate comprehensive reports for informed decision-making.
Engaged in the creation of SSIS jobs to automate report generation and cube refresh packages, streamlining data processing and enhancing reporting capabilities.
Proficient in SharePoint, leveraging expertise in managing and collaborating on documents, workflows, and team sites to enhance productivity and streamline business processes.
Resolved technical SQL issues by capturing traces through SQL Profiler in SQL Server.
Proficiently installed and upgraded higher education Campus Nexus CRM, integrated modules, meeting client needs.
Attracted Coppin University as a new client, showcasing effective CRM delivery and boosting Net Promoter Score (NPS).
Conducted functional testing, user acceptance testing (UAT) and performance testing to verify the client's needs are met.
Reviewed technical requirements and produced test strategies, test scenarios, and test cases to validate code that generated data in reports and dashboards.