Rishabh

Rishabh

Data Engineer
United States of America

About Me

Data Engineer with experience in building data pipelines, application support, and SQL optimization. Also experienced in data science and software development internships involving email classification, anomaly detection…

Experience

Senior Data Engineer

Larsen & Toubro Infotech
Jun 2021 - Present · 5 years

 Build data pipelines that ingested and transformed raw data of size 100 TB daily from different sources such as Oracle DB,
Postgres DB, MySQL DB and Flat files into optimized file formats like parquet, orc format.
 Managed critical Application Support, providing Technical L2 and L3 assistance in a collaborative team environment to
ensure uninterrupted operation of 100+ ETL applications for ABSA, South Africa's largest bank, with a proactive approach
to incident resolution and problem-solving.
 Led multiple optimizations and migrations in existing applications as part of service improvements, hence saved 2 FTEs
(Full-time Equivalent) and 120 hours per month of manual efforts.
 Modified and created complex business specific SQL scripts to leverage partitioning and CT tables to reduce query time by
several hours.
 Implemented MD5 Hashing mechanism in existing code to ensure 100% accuracy and completeness of source file and hence
helped client meet audit findings and regulations.

Data Engineer

LTIMindtree
Jun 2021 - Jul 2023 · 2 years 1 month

Certificate of Excellence 2022
Built data pipelines that ingested and transformed raw data of size 100 TB daily from different sources such as Oracle DB, Postgres DB, MySQL DB and Flat files into optimized file formats like parquet and orc format.
Managed critical Application Support, providing Technical L2 and L3 assistance in a collaborative team environment to ensure uninterrupted operation of 100+ ETL applications for ABSA, South Africa's largest bank, with a proactive approach to incident resolution and problem-solving.
Led multiple optimizations and migrations in existing applications as part of service improvements, hence saved 2 FTEs (Full-time Equivalent) and 120 hours per month of manual efforts.
Modified and created complex business specific SQL scripts to leverage partitioning and CT tables to reduce query time by several hours.
Implemented MD5 Hashing mechanism in existing code to ensure 100% accuracy and completeness of source file and hence helped client meet audit findings and regulations.

Data Scientist Intern

Netcore Solution
Feb 2020 - Feb 2021 · 1 year

Automated the classification of banking emails by creating a model and training it to identify different categories for example cards, loans, advertisement etc, hence reducing the manual efforts by 60%.
Created Python code to pre-process the input vector (the body of an email); included data collection, tokenization, stemming, conversion to desired upper or lower case, applying TF-IDF.
Designed Two layer Dense Neural network and trained model to perform two level classification of emails (first into the “Category” and second into “Product”) of banking sector.

Software Developer Intern

Gajsheild Infotech
Jun 2019 - Dec 2019 · 6 months

Constructed an Anomaly Detection algorithm utilizing Logistic Regression to raise an alert whenever an employee outside of its shift hours logs into the system hence improving the monitoring security of company by 25%.
Constructed a website using Web2Py framework to build charts and graphs for sales of the company hence providing clear insights to sales team of the company hence improving reporting speed by 50%.

Skills

Java Python SQL Database Apache Hadoop PySpark Scala MySQL MS SQL PostgreSQL Hive Bash Scripting Hadoop Apache Spark Informatica PowerCenter CA WADE Linux/Unix Airflow Snowflake AWS S3 AWS EC2 AWS Glue Oracle DB Parquet ORC SQL Tableau HDFS JSON CSV TF-IDF Tokenization Stemming Logistic Regression Web2Py Dense Neural Network
Report this Profile?