Around 3 years experience in requirement analysis, data pipeline development and debugging with pyspark, python and airflow along with database management.
Proficiency working with cloud technologies with working experie…
Around 3 years experience in requirement analysis, data pipeline development and debugging with pyspark, python and airflow along with database management.
Proficiency working with cloud technologies with working experience with AWS product and good knowledge with azure.
Extensive experience in working with various databases with working experience with SQL.
Have strong knowledge of algorithms and data structures.
Delivered requirements consistently working with agile team with expertise in using Jira.
Able to strong version control standards and working experience with GIT.
Python & Pyspark Developer November 2021 – May 2023
Tata Consultancy Services, India
• Involved in understanding user requirements and developing complex data pipelines comprising data sources such as file systems, On-prem cloud databases as well as cloud hosted databases collecting data from different applications.
• Implemented DataMart in production, scheduled jobs in airflow and debugged production failures and provided quick resolutions in event of pipeline failures.
• Created a robust automation script to validate data integrity across multiple loading strategies for data lake and DataMart, enabling efficient and accurate data validation for data analysts; saved 20+ hours per week and increased overall data quality.
• Developed and implemented a value-added project for clients, delivering comprehensive quality measurement; reduced manual data analysis tasks by 40% and saved 20 hours per week for analysts.
Ab Initio Developer & Production Admin October 2020 – October 2021 Tata Consultancy Services, India
• Streamlined data lineage process by conducting dependency analysis, resulting in enhanced understanding of data flow and improved version control practices; increased team efficiency by 40% and reduced errors by 50%
• Engineered Ab Initio graphs with a focus on performance enhancement, employing rollup, reformat, scan, and filters to meet client demands; executed thorough analysis of intricate graph structures to ensure seamless data processing and quality assurance.
• Monitored, managed, and fixed failure for daily batch jobs for Data Lake (S3) and DataMart (Redshift) using Control Center in short span of time.
• Trained a team of 15 resources in GDE, Unix, production handling, and server management, resulting in a 40% improvement in system uptime and a 30% reduction in average incident response time.
• Good knowledge of best practices project Architecture, server management and Unix Shell scripting.
Tableau Developer October 2020 – May 2023
Tata Consultancy Services, India
• Build dashboards for validations of data quality ingested in Data Lake and DataMart.
• Created custom data queries from multiple source tables using SQL Workbench to meet the reporting needs of the end users.
• Contributed to an agile team and managed tasks and documentation using project management tools Jira and Confluence.
Involved in understanding user requirements and developing complex data pipelines comprising data sources such as file systems, On-prem cloud databases as well as cloud hosted databases collecting data from different applications.
Implemented DataMart in production, scheduled jobs in airflow and debugged production failures and provided quick resolutions in event of pipeline failures.
Created a robust automation script to validate data integrity across multiple loading strategies for data lake and DataMart, enabling efficient and accurate data validation for data analysts; saved 20+ hours per week and increased overall data quality.
Developed and implemented a value-added project for clients, delivering comprehensive quality measurement; reduced manual data analysis tasks by 40% and saved 20 hours per week for analysts.
Build dashboards for validations of data quality ingested in Data Lake and DataMart.
Created custom data queries from multiple source tables using SQL Workbench to meet the reporting needs of the end users.
Contributed to an agile team and managed tasks and documentation using project management tools Jira and Confluence.
Streamlined data lineage process by conducting dependency analysis, resulting in enhanced understanding of data flow and improved version control practices; increased team efficiency by 40% and reduced errors by 50%.
Engineered Ab Initio graphs with a focus on performance enhancement, employing rollup, reformat, scan, and filters to meet client demands; executed thorough analysis of intricate graph structures to ensure seamless data processing and quality assurance.
Monitored, managed, and fixed failure for daily batch jobs for Data Lake (S3) and DataMart (Redshift) using Control Center in short span of time.
Trained a team of 15 resources in GDE, Unix, production handling, and server management, resulting in a 40% improvement in system uptime and a 30% reduction in average incident response time.
Good knowledge of best practices project Architecture, server management and Unix Shell scripting.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
