Nishikant Reddy

<p>Managing BAU tasks for daily running ETL pipelines, monitoring and raising and resolving tickets for run failures<br>
Requirements gathering to create High Level Design for ETL Pipelines in ADF for new data sources<br>
Engaging with users, stakeholders understang the Data sources, Transformations Business Rules needed to be applied to load data into destination & streamlining the design and data modelling in Kimball methodology<br>
Designing & developing ETL pipelines using ADF and migrating data from diseparate datasources to existing datawarehouse in Fabric environment<br>
Implementation of security and governance for end-to-end ETL pipeline solutions<br>
SQL development, performance tuning of stored procedures for querying, data transformation from source tables and DWH tables<br>
Testing and release management using CICD pipelines<br>
Streamlining business process and requirements analysis<br>
Automation of multidimensional data processing – development of ETL process feeding application with data coming from variety of data sources (using GitHub for version control, creating package for handling functions, implementing unit testing and mail alerting for handling errors)<br>
Improved performance and automated of the solutions using AI tech.<br>
Collaborate with business and technical stakeholders to define data requirements and deliver fit-for-purpose solutions.</p>

<p>To develop and maintain the Common Logical Data Model and the physical data models for the System of Record, Data Marts under direction from the Senior Data Engineer and Data Architect<br>
To develop and maintain the specifications for extract, transform and load (ETL) routines that enable data to be ingested into the System of Record or distributed across the data marts, to support wider analytics or BI dashboards and visualizations.<br>
Lead the design and delivery of robust and scalable ETL solutions across multiple data sources and platforms.<br>
Architected and implemented a cloud-native, serverless Airflow and AI infrastructure on AWS, reducing operational costs by 40% and improving pipeline reliability to 99.99% uptime<br>
Designed and deployed an end-to-end data lakehouse architecture integrating AWS S3 and Snowflake, scaling data ingestion capacity by 150%.<br>
Configured Snowpipe alongside AWS SQS event notifications to automate near-real-time ingestion of 10M+ daily events from mobile applications.<br>
Develop analytical insights that influence business efficiencies.<br>
To test the data warehouse front-end visualization tools. Support the MIS team to ensure that all visualizations meet the business requirements and adhere to business BI standards.<br>
Integrating master data and utilizing it in the datawarehouse<br>
Re-architected a legacy batch-based ETL pipeline into a real-time event-driven streaming system using Apache Flink and Kafka<br>
Use appropriate agile techniques and tools, ensuring traceability of user needs to source, and adherence to business objectives.<br>
Assist decision makers by gathering, summarizing and presenting information to help them make informed decisions.<br>
Influence stakeholders and manage relationships effectively by communicating and engaging frequently.<br>
Understand and communicate the ‘big picture’, continually ensuring that product goals are adhered to.<br>
Work at a detailed level during sprints and product iterations</p>

<p>Merging complex data sets with SQL development<br>
Understanding good standards of QA (i.e. row count reconciliation when performing joins)<br>
Data Profiling and data cleansing using SQL stored procedures<br>
Building reports in Tableau and conveting key metrics to usable reports<br>
Liaising with BAs to gather requirements for reports<br>
Data quality analysis with data users.<br>
Working in an Agile environment and providing regular updates regarding tickets assigned.<br>
Migration of the app environment to AWS along with rewriting all R codes into Python</p>

<p>Working with and designing a data platform, and involved in data migration, migrating data from new vendors systems to existing datawarehouse<br>
Implementing data modeling principles, including data lakes, and warehousing tools<br>
Implementation of end-to-end DWH and ETL pipeline solutions, and converting data into flat file format<br>
Maintenance and optimization of Data Warehouse solutions to maximize performance<br>
Developing Azure DataBricks pipelines for data ingestion in Mircosoft Fabric<br>
SQL development, writing SQL code for tables, stored procedures, views.<br>
Testing and software release management<br>
Streamlining business process and requirements analysis<br>
Automatizing of multidimensional data processing – development of ETL process feeding a Shiny application with data coming from variety of data sources (in R on Linux; using Bitbucket for version control, creating package for handling functions, implementing unit testing and mail alerting for handling errors)<br>
Integrating master data from different data sources into MDM services.</p>

<p>Using the Microsoft BI stack in Mircosoft Fabric to analyze structured and unstructured data<br>
Ensuring data quality is maintained throughout the process.<br>
Lead the design and delivery of robust and scalable ETL solutions across multiple data sources and platforms using Azure Datafactory and Azure Databricks<br>
Automated ingestion into Snowflake via Snowpipe, transforming the raw JSON schemas into analytical tables using Snowpark Python.<br>
Collaborate with business and technical stakeholders to define data requirements and deliver fit-for-purpose solutions.<br>
Develop and maintain both real-time and batch data processing pipelines to support analytics and reporting needs.<br>
Ensure data quality, consistency, and integrity across all systems and processes.<br>
Development of database objects using T-SQL (stored procedures, tables, functions etc.)<br>
Develop reports using PowerBI<br>
Ongoing development of the ADF pipelines to load data into snowflake Datawarehouse.<br>
Using CI/CD approach for continuous development and used Devops pipeline for deployment.<br>
Engage with other technical and non-technical teams to provide unified solutions.<br>
Respond to technical and non-technical queries from all areas of the business<br>
Managed & lead a team of 4 people, & reported to BI Manager</p>

<p>Taking ownership of master data integrity in key systems and assist in the application procedures and migrating master data from various sources into MDM system<br>
Work on implementation procedures of data standards and guidelines on data ownership, coding structures, and data replication to ensure access to and integrity of data sets.<br>
Maintained and developed PowerBI reports, used DAX to pull data.<br>
Working closely with Data Quality team to identify areas for data quality improvements<br>
Resolving data problems through the appropriate choice of error detection and correction, process control and improvement, or process design strategies.<br>
Data modeling, data cleansing, and data enrichment<br>
Managed & lead a team of 3 people, and reported to Head of Data</p>

<p>Data analysis, SQL and ETL solutions development for Data Migration project<br>
Migrating Master Data from Legacy payment Systems Dolphin and Hybris5 to Hybris6<br>
Migrating operational data, master data to SAP systems Hybris6.<br>
Writing SQL code to consolidate customer data, Sales Orders, and payments transactions<br>
Developing ETL solutions to load data from Legacy systems to Hybris using SSIS and Webmethods, and migrating data & producing flat files<br>
Developing pipelines with DataBricks to load data into Azure Data Lake and Azure SQL Datawarehouse.<br>
Developed streaming ETL utilizing Apache Flink DataStream API to ingest, clean, and enrich clickstream data<br>
Built a serverless streaming pipeline fetching real-time stock API data using AWS Lambda and landing it in Amazon S3.<br>
Automated ingestion into Snowflake via Snowpipe, transforming the raw JSON schemas into analytical tables using Snowpark Python.<br>
Designing and created PowerBI reports to replace existing excel reports.<br>
End to end data reconciliations<br>
Liaising with users and teams across the board to gather requirements and mitigate risks.</p>

<p>Migrating Client data into the Datawarehouse and Data analysis.<br>
Create new and maintain existing ETL solutions.<br>
Data extraction, transformation and loading from diverse data sources flat files, excel files, and SQL Server 2016.<br>
Data injection using DataFactory into the Azure DataLake<br>
Query optimization, SSIS packages performance tuning.<br>
Reconciliation reports developed using DataBricks and PowerBI, automated the process to generate reports in excel files.<br>
Documentation for the development and changes applied.<br>
Providing regular updates to management and business users on status of tasks.<br>
Requirements gathering, conducting meetings in coordination with the stakeholders.</p>

<p>Design and deliver ETL packages to implement the functional specification of solutions.<br>
Conduct unit testing and troubleshooting.<br>
Carrying out peer reviews with colleagues.<br>
Deliver ETL solutions to established development standards and practices.<br>
Attend daily scrum meetings with the development team, weekly and monthly project team meetings as appropriate.<br>
Implemented Business Intelligence / Data warehouse solution using Kimball methodology<br>
Optimize existing data flows<br>
Design, build and deploy BI solutions in SSIS<br>
Data injection - Build data pipelines to inject data from a wide range of source systems. Maintains and streamlines data pipeline architectures.<br>
Designing and creating PowerBI reports, used DAX to pull data.<br>
Designs test cases and test scripts under own direction, mapping back to predetermined criteria, recording and reporting outcomes.</p>

<p>Developed, deployed, and monitored SSIS Packages for new ETL Processes and upgraded the existing DTS packages to SSIS for the on-going ETL Processes.<br>
Monitored Full/Incremental/Daily Loads and support all scheduled ETL jobs for batch processing.<br>
Used different containers in SSIS packages to perform various ETL tasks.<br>
Created SSIS templates for developing SSIS packages in such a way that they can be dynamically deployed into development, testing and production environments.<br>
Involved in Technical decisions for Business requirement, Interaction with Business Analysts, Client team, Development team, Capacity planning and Up gradation of System Configuration.<br>
Writing stored procedures, SQL functions, UDF’s and Triggers and applying changes to the existing ones.<br>
Designing, developing Cube measure and dimension using SSAS<br>
Writing MDX queries to retrieve data from cubes, to define calculated members<br>
Developed custom calculated measures using DAX in PowerBI.<br>
Developing and maintaining OLAP databases.<br>
Involved in Installation, Configuration and Deployment of Reports using PowerBI with Scale-Out topology.<br>
Designing and building data pipelines and data streams.<br>
Design and build data service APIs.<br>
Analyzing current business practices, processes and procedures and identifying future opportunities for leveraging Microsoft Azure data & analytics PaaS services.<br>
Recreating existing application logic and functionality in the Azure Data Lake, Data Factory, and SQL Database environment.<br>
Designing and maintaining database schemas.<br>
Build complex data pipelines architecture in Azure datafactory in projects which develop new BI solutions</p>

<p>Worked in Global Payments Change on various projects, worked on Payments in Error to develop database for internal application, Migration from SQL Server 2005 to SQL Server 2008R2, CASS(client asset), internal debt collection reconciliation for PPI, internal data analysis project for NPA accounts, Nostro reports using Crystal reports, MI database creation for Core system Mapping<br>
Designing and creating database using SQL Server 2008R2<br>
Writing stored procedures, SQL functions, UDF’s and Triggers and applying changes to the existing ones.<br>
Developed reports using PowerBI and used table control, matrix control for cross tab reports.<br>
Rendering reports, managing subscriptions, creating reports with report builder, report snapshot and report cache.<br>
Scheduled reports for daily, weekly, monthly reports for executives, Business analysts and customer representatives for various categories based on business needs using PowerBI.<br>
Create and maintain ETL packages<br>
Data extraction, transformation, and loading, involving flat files, excel files and SQL Server 2008 R2 tables as data sources and destinations.<br>
Developing data migration scripts and migrating all the dependencies (reports, SSIS packages)<br>
Creating database backups and restoring it on another server<br>
Creating database and queries for PPI project for internal analysis of the data, creating database and queries in SQL Server Management Studio 2008R2 for NPA accounts analysis<br>
Developing reports in Crystal reports using Oracle 11g as the data source.<br>
Reconciliation reports developed using PowerBI, automated the process to generate reports in excel files.<br>
Supporting the CASS database and resolving tickets raised<br>
Query optimization, SSIS packages performance tuning.<br>
Documentation for the development and changes applied.<br>
Providing regular updates to management and business users on status of tasks.<br>
Requirements gathering, conducting meetings in coordination with the stake holders.</p>

<p>Work on BI projects including resource quotes, scoping of requirements, database / cube design & build, report creation, testing, documentation etc<br>
Investigate, recommend, and develop appropriate reporting and BI technique<br>
Ensure full compliance with Change / Problem / Project Management processes and the development lifecycle<br>
Attend weekly team meetings to understand priorities and communicate progress to the Global IT Reporting Manager<br>
Investigate, recommend, and develop appropriate reporting techniques<br>
Provide 2nd & 3rd line support of existing reporting and BI systems<br>
Create and maintain knowledge-base and solution documentation, as well as end-user guides<br>
Support existing business stakeholder relationships<br>
Look for opportunities to improve reporting and BI quality, consistency, and functionality<br>
Ensure all Data Protection Principles and PCI DSS requirements are complied with through all reporting and BI systems</p>

<p>Closing defects on a high priority project<br>
Fault-finding and troubleshooting existing T-SQL code (SQL Server 2008R2)<br>
Support and enhancements of SQL code, used GIT<br>
PowerShell scripting<br>
Used HP ALM to manage and update the defects.</p>

<p>SQL development in SQL Server 2008R2 and Oracle 11g, developing SSIS packages and writing test scripts in C# required for data migration.<br>
Writing stored procedures, SQL functions, UDF’s and Triggers and applying changes to the existing ones.<br>
Create and maintain ETL packages<br>
Data extraction, transformation and loading, involving flat files, excel files, Oracle tables and SQL Server 2008 R2 tables as data sources and destinations.<br>
Query optimization, SSIS packages performance tuning.<br>
Data reconciliation at each load stage of the migration phase.<br>
Reconciliation reports developed for Finance and HR data, automated the process to generate reports in excel files.<br>
Developing unit tests using C# in MS Visual Studio 2010.<br>
Installation of Oracle tools required for development and connectivity.<br>
Documentation for the development and changes applied.<br>
Providing regular updates to management and business users on status of tasks.<br>
Requirements gathering, conducting meetings in coordination with the stakeholders.<br>
Wrote and managed SQL stored procedures and queries on SQL Server 2012 deployed in Amazon RDS</p>

<p>Creation and tuning of current and new data warehouses<br>
Create and Maintain ETL packages between MySQL and MS SQL<br>
Perform DBA Functions between various databases<br>
Creating Tables, Views, and non-clustered indexes in SQL Server 2008R2 and 2012<br>
Writing Stored Procedures, UDF’s, Triggers and T-SQL queries<br>
Normalization of data<br>
Designing, developing Cube measure and dimension.<br>
Writing MDX queries to retrieve data from cubes, to define calculated members<br>
Writing MDX scripts to define calculations<br>
Develop Report Structure with data definition, report layout and report definition language.<br>
Performing optimization and automation while undertaking tasks and providing production support.<br>
Validate and monitor data between databases and create KPI presentations to manage monitoring.<br>
Kimball Methodology used in datawarehousing<br>
Documentation and organization of completed tasks in such a manner that they can easily be referred to in the future.<br>
Providing regular updates to management and business owners on status of tasks<br>
Work closely with other members of the BI team, offering suggestions for optimization of the technical systems.</p>

<p>Data manipulation, developing new data structures, and dealing with transactional data, to deduce the value of the customer base.<br>
Creating Tables, Views, and non-clustered indexes in SQL Server 2005/2008<br>
Writing Stored Procedures, T-SQL queries, User Defined Functions.<br>
Data normalization.<br>
Designed, developed, and deployed SSIS packages<br>
Created and scheduled jobs using SQL Server Agent<br>
Involved in complex data models<br>
Dealt with network data which was used across business for promotions to customers.<br>
Liaised with key stakeholders across the business to communicate back the results from initial analysis and provided commercial objectives and benefits to the board level.</p>

<p>Developing reports in LogiXML using SQL Server 2008/R2.<br>
Writing required SQL queries, stored procedures, and Transactions for the reports.<br>
Optimizing query by creating various clustered, non-clustered indexes and indexed views, to make reports execution faster.<br>
Did analysis for dashboards and scorecards<br>
Worked closely with the existing MIS/Business team to ensure the groups provide the best possible end-to-end reporting solution.</p>

<p>Implementation and delivery of BI platform solutions to develop and deploy ETL, analytical, reporting and scorecard\dashing on SQL Server 2008 using SSIS/DTS, SSAS, SSRS.<br>
Used SSIS to create ETL packages to validate, extract, transform and load data to data warehouse databases.<br>
Involved in daily loads (full & incremental) into staging, troubleshooting process, issues and errors using SQL Server Integration Services (SSIS) 2008.<br>
Extracted and reviewed data from heterogeneous sources (HTML files, text files, Excel, and Access Files) from OLTP to OLAP using SQL Server Integration Services (SSIS) 2008.<br>
Applied various data transformations like SCD (Slowly Changing Dimension) - to maintain the history of the data, aggregate, sort, multicasting, conditional split, and derived column.<br>
Created stored procedures/functions and used them in SSRS to generate ad-hoc reports.<br>
Generated reports using global variables, expressions, and functions for the reports.<br>
Developed reports using SSRS and used table control, matrix control for cross tab reports.<br>
Created and Configured Data Source & Data Source Views, Dimensions, Cubes, Measures, Partitions, KPI’s & MDX Queries using SQL Server 2008/2005 Analysis Services.<br>
Scripted OLAP database backup and scheduled a daily backup using SQL Server agent job.</p>

<p>Developed, deployed, and monitored SSIS Packages for new ETL Processes and upgraded the existing DTS packages to SSIS for the on-going ETL Processes.<br>
Monitored Full/Incremental/Daily Loads and support all scheduled ETL jobs for batch processing.<br>
Used different containers in SSIS packages to perform various ETL tasks.<br>
Created SSIS templates for developing SSIS packages in such a way that they can be dynamically deployed into development, testing and production environments.<br>
Involved in Technical decisions for Business requirement, Interaction with Business Analysts, Client team, Development team, Capacity planning and Up gradation of System Configuration.<br>
Involved in Installation, Configuration and Deployment of Reports using SSRS with Scale-Out topology.<br>
Rendering reports, managing subscriptions, creating reports with report builder, report snapshot and report cache.<br>
Scheduled reports for daily, weekly, monthly reports for executives, Business analyst and customer representatives for various categories based on business needs using SQL Server Reporting Services (SSRS).<br>
Responsible for scheduling the subscription reports with the subscription report wizard.<br>
Schedule jobs and alerting using SQL Server Agent.</p>

<p>Involved in SQL Server Configuration, Administration, Implementation and Troubleshooting for Business work.<br>
Performed Physical and Logical Data Modeling for databases of varied sizes.<br>
Performed Regular Database Maintenance process, checking the disk Defrag procedures in different environments.<br>
Involved in Installation, Configuration and Deployment of SSRS Reports using different topologies.<br>
Involved in Creation, Development and Deployment of SSIS packages in SQL Server 2005.<br>
Designed calculated measures and dimension members using multi-dimensional expression (MDX), mathematical formulas and user-defined functions.<br>
Provided On-Call support to Mission critical Environments and expertise with Disaster Recovery mechanisms.<br>
Involved in training Update features for Business solutions of SQL Server.<br>
Used SQL Profiler for Performance monitor to resolve DeadLocks and long running queries by checking appropriate changes to Transaction Isolation levels.</p>