Sundus Hasan

Sundus Hasan

Azure Data Engineer
India

نبذة عني

Azure Data Engineer with 5 years of experience at Tata Consultancy Services, currently with Hexaware Technologies Limited, specializing in designing, developing, and supporting scalable data pipelines using Python, SQL, …

الخبرة

Data Engineer

Hexaware Technologies Limited
Aug 2026 - حتى الآن · 2 أشهر

<p>Technology/Skills: Azure Data Factory (ADF), Azure Databricks, PySpark, Spark SQL, Python, SQL, Azure Data Lake Storage (ADLS Gen2), Delta Lake<br>
Project: AmeriHealth – Enterprise Data Lake Optimization<br>
Currently working on Enterprise Data Lake Optimization, focusing on improving pipeline performance, scalability, reliability, and cost efficiency.<br>
Analyze data pipelines, Spark jobs, stages, and tasks to identify performance bottlenecks and optimization opportunities.<br>
Evaluate data volume, partition size, partition count, repartition/coalesce, shuffle read/write, data skew, execution time, and cluster utilization to identify resource and performance gaps.<br>
Identify and recommend optimization techniques including partition optimization, repartition/coalesce, data-skew handling, shuffle reduction, join optimization, caching, and appropriate cluster sizing.<br>
Analyze CPU, memory, executor utilization, job duration, and resource consumption to improve cluster utilization and reduce infrastructure costs.<br>
Collaborate with teams to implement and validate optimization solutions by comparing before-and-after performance and cost metrics.<br>
Attended a training on Building AI Solutions – Generative AI with LLMs, to learn how Generative AI and Large Language Models (LLMs) are used to build AI solutions.</p>

Agentic AI Engineer Intern

Qwala
Mar 2026 - Jul 2026 · 4 أشهر

<p>Technology/Skills: Python, Agentic AI, Generative AI, Multi-Agent Architecture, Ollama, Gemma, Qwen3, SQLite, SQL, Prompt Engineering, Memory Management<br>
Project: Children AI Companion<br>
Designed and developed a multi-agent AI-powered learning companion for nursery-to-primary children using Python, Ollama, Gemma, and Qwen3.<br>
Built an intelligent query-routing system with specialized agents for Mathematics, Alphabet, Phonics, Quiz, Poems, Science, and Study Assistance, combining lightweight agents for simple tasks with local LLMs for complex queries.<br>
Implemented SQLite-based conversation memory and learning-progress tracking to support personalized learning.<br>
Added age-appropriate safety controls to ensure a secure, child-friendly learning experience.</p>

Big Data Engineer

Tata Consultancy Services
Aug 2018 - May 2023 · 4 سنوات 9 أشهر

<p>Client 1: State Bank of India (SBI), Mumbai, India<br>
Designation: Assistant System Engineer<br>
Technology/Skills: Linux (Red Hat 7.5), Azure DevOps (Git – CI/CD), SQL<br>
Performed data analysis and validation using SQL on large banking datasets to support operational and business reporting.<br>
Worked on Linux-based (Red Hat Enterprise Linux) servers to access data files, logs, and execute analytical queries.<br>
Developed and maintained shell scripts for batch job monitoring, file availability checks, and alert generation.<br>
Monitored daily data pipelines and batch executions using Dynatrace, identifying job delays and data availability issues.<br>
Investigated data discrepancies and supported root-cause analysis for production incidents.<br>
Coordinated with upstream and downstream teams to ensure timely data delivery and SLA compliance.<br>
Used Azure DevOps (Git) for version control and CI/CD to deploy scripts and configuration changes.<br>
Prepared ad-hoc reports and analysis based on business requirements.<br>
Used JIRA and Confluence for defect tracking, documentation, and knowledge sharing.<br>
Client 2: ABN AMRO Bank, Amsterdam, Europe<br>
Designation: System Engineer<br>
Technology/Skills: Python, SQL, PySpark, Azure DevOps, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Apache Airflow, Azure Databricks, Delta Lake, Unity Catalog, Jira, Confluence, Agile Methodologies<br>
Delivered data engineering solutions supporting Financial Crime Detection, Fraud Detection, and KYC (Know Your Customer) initiatives.<br>
Designed, developed, and maintained end-to-end data pipelines using Python, SQL, Spark, and PySpark to process large-scale structured data.<br>
Built and managed ETL workflows using Azure Data Factory (ADF) for ingesting data from multiple source systems into Azure Data Lake Storage (ADLS Gen2).<br>
Implemented data transformation and aggregation logic using Azure Databricks notebooks integrated with Apache Spark, storing curated datasets as Delta Lake tables for ACID-compliant, versioned data storage.<br>
Configured and managed data access controls and governance using Unity Catalog within Databricks, ensuring secure and auditable data usage across teams.<br>
Designed data models to support analytics and reporting requirements across ingestion and curation layers.<br>
Developed incremental and batch data processing frameworks to improve performance and reduce data processing time.<br>
Orchestrated and scheduled data pipelines using Apache Airflow, handling dependencies, retries, alerts, and SLA monitoring.<br>
Optimized Spark jobs by applying partitioning, caching, and efficient join strategies, improving pipeline performance and reliability.<br>
Used Git for version control and collaborated with teams through Azure DevOps for CI/CD pipelines and release management.<br>
Performed data validation, reconciliation, and error handling to ensure data quality and consistency across layers.<br>
Actively supported production deployments and incident resolution, identifying root causes and implementing long-term fixes.<br>
Worked in an Agile environment, participating in sprint planning, daily stand-ups, sprint reviews, and retrospectives.<br>
Used JIRA for task and defect tracking and Confluence for technical documentation and knowledge sharing.<br>
Collaborated with business analysts, QA teams, and stakeholders to translate business requirements into scalable data solutions.</p>

المهارات

Python SQL Apache Spark Spark SQL PySpark Delta Lake Azure Data Factory Azure Data Lake Storage (ADLS Gen2) Azure Databricks Unity Catalog Azure DevOps ETL/ELT Pipeline Design Data Validation & Reconciliation SQL Server Apache Airflow Git GitHub Linux Windows Agentic AI Generative AI Multi-Agent Architecture Ollama Gemma Qwen3 SQLite Prompt Engineering Memory Management
الإبلاغ عن هذا الملف الشخصي؟