A highly skilled and motivated Data Engineer with 12+ years of experience in designing and implementing data architecture, building data pipelines, and optimizing data processing systems. Seeking opportunities to contrib…
A highly skilled and motivated Data Engineer with 12+ years of experience in designing and implementing data architecture, building data pipelines, and optimizing data processing systems. Seeking opportunities to contribute my expertise in data engineering to drive data-driven decision-making and enhance data-driven applications.
Strong proficiency in managing and generating insights from large volumes of data using Pyspark, Alteryx/ SQL/Python, and presenting them through visualization tools (Tableau / Power BI/QlikView)
• Experience in building data pipelines and data visualization using Azure cloud services such as Azure Data Factory, Azure Data Bricks, Azure Synapse and Power BI
• Experience in CI/CD {Continuous Integration and Deployment} for code builds and deployments of Data pipelines.
• Profound experience in performing data ingestion, Data processing (Transformation, enrichment and aggregation)
• Expert in designing parallel jobs using various stages like join, merge, lookup, remove duplicates, filter, complex flat file, aggregator.
• Experienced with JSON based RESTful web services and XML based SOAP web services and also worked on various applications using python integrated IDE’s like PyCharm and Databricks.
• Excellent understanding of Medallion and Publisher/Subscriber architecture for delivering Data Integration applications using Delta Lake and Lake house.
• Highly experienced in Relational and dimensional modelling (Star and Snowflake Schema)
• Proven track record of delivering high-quality and scalable data solutions on time and within budget
• Experience in liaising with stakeholders including business Management, operational and testing teams.
• Highly experienced in Ab into batch and continuous graphs, Plans, Metadata hub, TRMC, control centre and performance tuning
• Hands on experience with Amazon Ec2, Amazon s3, Amazon RDS, IAM, amazon Elastic Load Balancing, Auto scaling, cloud Watch, SNS, SES, SQS, Lambda, EMR and other services of AWS family.
• Technology used – PySpark (Python 2.7), Oracle, Teradata, MongoDB, Hadoop, Bitbucket, On-premise Unix Servers, Tableau etc.
ACHIEVEMENTS:
• Process Automation for DQ Check: Developed and deployed an automated process for data quality (DQ) checks across multiple banking data sources. This automation improved data accuracy and consistency, reducing manual effort by 70% and ensuring high-quality data for critical banking operations and reporting.
• Developed and optimized a series of Python-based ETL pipelines to process and integrate transactional data from multiple banking systems. This led to a 30% increase in data processing speed, ensuring timely and accurate financial reporting.
• Batch Monitoring (Feed / Feed Completion Status) : Implemented ETL batch monitoring and batch forecasting to enhance ETL process efficiency. Developed automated systems for real-time tracking and prediction of ETL workflows, improving reliability and reducing downtime. This innovation optimized resource allocation, ensured ti
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
