نبذة عني
Azure Data Engineer with 5 years of experience at Tata Consultancy Services, currently with Hexaware Technologies Limited, specializing in designing, developing, and supporting scalable data pipelines using Python, SQL, …
Azure Data Engineer with 5 years of experience at Tata Consultancy Services, currently with Hexaware Technologies Limited, specializing in designing, developing, and supporting scalable data pipelines using Python, SQL, Spark, and PySpark. Hands-on experience with Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Azure Databricks for building cloud-based ETL solutions, including Delta Lake tables for reliable, versioned data storage and Unity Catalog for data governance and access control. Worked with multiple banking clients including State Bank of India (Mumbai) and ABN AMRO (Europe), delivering reliable data solutions in Agile environments. Proficient in workflow orchestration using Apache Airflow, version control with Git/GitHub, CI/CD using Azure DevOps, and Agile methodologies, with active participation in sprint planning and production support. Also gained exposure to Agentic AI system design through an internship at Qwala (March–July 2026), building multi-agent AI solutions using Python, Ollama, Gemma, and Qwen3.
الخبرة
Data Engineer
<p>Technology/Skills: Azure Data Factory (ADF), Azure Databricks, PySpark, Spark SQL, Python, SQL, Azure Data Lake Storage (ADLS Gen2), Delta Lake<br>
Project: AmeriHealth – Enterprise Data Lake Optimization<br>
Currently working on Enterprise Data Lake Optimization, focusing on improving pipeline performance, scalability, reliability, and cost efficiency.<br>
Analyze data pipelines, Spark jobs, stages, and tasks to identify performance bottlenecks and optimization opportunities.<br>
Evaluate data volume, partition size, partition count, repartition/coalesce, shuffle read/write, data skew, execution time, and cluster utilization to identify resource and performance gaps.<br>
Identify and recommend optimization techniques including partition optimization, repartition/coalesce, data-skew handling, shuffle reduction, join optimization, caching, and appropriate cluster sizing.<br>
Analyze CPU, memory, executor utilization, job duration, and resource consumption to improve cluster utilization and reduce infrastructure costs.<br>
Collaborate with teams to implement and validate optimization solutions by comparing before-and-after performance and cost metrics.<br>
Attended a training on Building AI Solutions – Generative AI with LLMs, to learn how Generative AI and Large Language Models (LLMs) are used to build AI solutions.</p>
Agentic AI Engineer Intern
<p>Technology/Skills: Python, Agentic AI, Generative AI, Multi-Agent Architecture, Ollama, Gemma, Qwen3, SQLite, SQL, Prompt Engineering, Memory Management<br>
Project: Children AI Companion<br>
Designed and developed a multi-agent AI-powered learning companion for nursery-to-primary children using Python, Ollama, Gemma, and Qwen3.<br>
Built an intelligent query-routing system with specialized agents for Mathematics, Alphabet, Phonics, Quiz, Poems, Science, and Study Assistance, combining lightweight agents for simple tasks with local LLMs for complex queries.<br>
Implemented SQLite-based conversation memory and learning-progress tracking to support personalized learning.<br>
Added age-appropriate safety controls to ensure a secure, child-friendly learning experience.</p>
Big Data Engineer
<p>Client 1: State Bank of India (SBI), Mumbai, India<br>
Designation: Assistant System Engineer<br>
Technology/Skills: Linux (Red Hat 7.5), Azure DevOps (Git – CI/CD), SQL<br>
Performed data analysis and validation using SQL on large banking datasets to support operational and business reporting.<br>
Worked on Linux-based (Red Hat Enterprise Linux) servers to access data files, logs, and execute analytical queries.<br>
Developed and maintained shell scripts for batch job monitoring, file availability checks, and alert generation.<br>
Monitored daily data pipelines and batch executions using Dynatrace, identifying job delays and data availability issues.<br>
Investigated data discrepancies and supported root-cause analysis for production incidents.<br>
Coordinated with upstream and downstream teams to ensure timely data delivery and SLA compliance.<br>
Used Azure DevOps (Git) for version control and CI/CD to deploy scripts and configuration changes.<br>
Prepared ad-hoc reports and analysis based on business requirements.<br>
Used JIRA and Confluence for defect tracking, documentation, and knowledge sharing.<br>
Client 2: ABN AMRO Bank, Amsterdam, Europe<br>
Designation: System Engineer<br>
Technology/Skills: Python, SQL, PySpark, Azure DevOps, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Apache Airflow, Azure Databricks, Delta Lake, Unity Catalog, Jira, Confluence, Agile Methodologies<br>
Delivered data engineering solutions supporting Financial Crime Detection, Fraud Detection, and KYC (Know Your Customer) initiatives.<br>
Designed, developed, and maintained end-to-end data pipelines using Python, SQL, Spark, and PySpark to process large-scale structured data.<br>
Built and managed ETL workflows using Azure Data Factory (ADF) for ingesting data from multiple source systems into Azure Data Lake Storage (ADLS Gen2).<br>
Implemented data transformation and aggregation logic using Azure Databricks notebooks integrated with Apache Spark, storing curated datasets as Delta Lake tables for ACID-compliant, versioned data storage.<br>
Configured and managed data access controls and governance using Unity Catalog within Databricks, ensuring secure and auditable data usage across teams.<br>
Designed data models to support analytics and reporting requirements across ingestion and curation layers.<br>
Developed incremental and batch data processing frameworks to improve performance and reduce data processing time.<br>
Orchestrated and scheduled data pipelines using Apache Airflow, handling dependencies, retries, alerts, and SLA monitoring.<br>
Optimized Spark jobs by applying partitioning, caching, and efficient join strategies, improving pipeline performance and reliability.<br>
Used Git for version control and collaborated with teams through Azure DevOps for CI/CD pipelines and release management.<br>
Performed data validation, reconciliation, and error handling to ensure data quality and consistency across layers.<br>
Actively supported production deployments and incident resolution, identifying root causes and implementing long-term fixes.<br>
Worked in an Agile environment, participating in sprint planning, daily stand-ups, sprint reviews, and retrospectives.<br>
Used JIRA for task and defect tracking and Confluence for technical documentation and knowledge sharing.<br>
Collaborated with business analysts, QA teams, and stakeholders to translate business requirements into scalable data solutions.</p>