Nitin kumar gahlawat

Nitin kumar gahlawat

Data Engineer
India
English
Full-time Work from Home

About Me

Results-driven Data Engineer with over 5+ years of experience specializing in building scalable data solutions across Azure, AWS, Microsoft Fabric, and Databricks. Proven expertise in designing end-to-end ETL/ELT pipelin…

Experience

Analyst - RNA

WNS · Gurgaon , Haryana , India
Dec 2024 - Present · 1 year 11 months

<p>Migrated complex, high-volume business queries to Microsoft Fabric, developing optimized stored procedures that significantly reduced report generation time and enhanced data accessibility.</p><p> Engineered and maintained mission-critical data pipelines, resolving complex SQL join fan-out issues and ensuring accurate cumulative calculations for core reporting.</p><p> Collaborated with cross-functional stakeholders to deliver comprehensive, granular business metrics and KPI dashboards for annual performance reviews.</p><p> Designed and implemented AWS-based data pipelines to extract data from Amazon Redshift and load it into Amazon S3, powering downstream analytics.</p><p> Constructed a configuration-driven AWS Glue ETL framework orchestrated via Step Functions, supporting full and incremental data loads to improve scalability.</p><p> Implemented metadata-driven pipeline architectures using reusable PySpark scripts, dynamically handling multiple tables and significantly reducing the development lifecycle.</p>

Data Engineer

ATF Labs · Agra , Uttar Pradesh , India
Apr 2023 - Dec 2024 · 1 year 7 months

<p>Architected and managed robust ETL pipelines using Python and PySpark for scalable ingestion and processing of large datasets from disparate sources.</p><p> Developed streaming applications using PySpark Structured Streaming to provide real-time calculations, improving alerting capabilities and system responsiveness.</p><p> Optimized complex SQL queries for data analysis and aggregation, ensuring fast and accurate retrieval of analytical datasets.</p><p> Automated Azure infrastructure provisioning using Terraform, deploying Databricks workspaces, clusters, and SQL Warehouses.</p><p> Established CI/CD pipelines in Azure DevOps, facilitating the smooth, automated deployment of data pipelines and infrastructure components.</p>

Programmer Analyst Trainee

Cognizant · Gurgaon , Haryana , India
Jul 2022 - Apr 2023 · 9 months

<p>Engineered a generic, metadata-driven data quality framework using PySpark to standardize and validate data via JSON configuration files.</p><p> Built cloud-native ETL pipelines utilizing AWS, SQL, Python, and PySpark to process high-volume enterprise data.</p><p> Automated secure data ingestion from SFTP to AWS S3 buckets utilizing Paramiko and Boto3 Python libraries.</p><p> Managed source code within GitHub repositories and configured Jenkins pipelines to seamlessly migrate code across testing and production environments.</p>

System Engineer Trainee

Infosys · Chandigarh , Chandigarh , India
Dec 2021 - Jun 2022 · 6 months

<p>Developed and orchestrated data pipelines in Azure Data Factory (ADF) to extract data from multiple sources and load it into Azure Synapse Analytics.</p><p> Transformed complex datasets using PySpark and orchestrated the loading of processed data into Azure Data Lake Storage (ADLS).</p><p> Built a comprehensive data warehouse incorporating diverse facts and dimensions, executing data transformations with Spark SQL.</p><p> Authored PySpark functions in Jupyter for rigorous data validation, including duplicate record elimination and type validation.</p>

Certifications

Microsoft Certified: Azure Database Administrator Associate

Microsoft Certified · Pilani , Rajasthan , India · 2026

AWS Certified Data Engineer - Associate

amazon · Pilani , Rajasthan , India · 2026

Skills

Python SQL PySpark AWS Azure Azure Data Factory (ADF) Microsoft Fabric AWS S3 Redshift AWS Glue AWS Step Function AWS Lambda Databricks RDS ETL Spark Snowflake dbt Apache Airflow Oracle OneLake Git VS Code Apache Superset Power BI Jenkins CI/CD Azure DevOps (ADO) Terraform Structured Streaming Spark SQL Jupyter Paramiko Boto3 Azure Synapse Analytics Azure Data Lake Storage (ADLS) GitHub
Report this Profile?