Senior AI Engineer with experience in big data processing, ETL, AI/ML automation, and data validation. Worked with Apache Spark, Hadoop, Hive, PySpark, Scala, Python, Docker, Kubernetes, and CI/CD tools to build and prod…
Senior AI Engineer with experience in big data processing, ETL, AI/ML automation, and data validation. Worked with Apache Spark, Hadoop, Hive, PySpark, Scala, Python, Docker, Kubernetes, and CI/CD tools to build and productionize data and machine learning solutions.
Developing high performance, distributed computing tasks for massive data processing using Big Data technologies including Apache spark, Hadoop, NoSQL, SQL and other distributed computing technologies
Developed high performance, distributed ETL tasks for massive data processing using Big Data technologies including Apache spark, Hadoop, NoSQL, Hive, Pyspark and other distributed computing technologies.
Scaled and productionized AI/machine learning solutions using on premise infrastructure and open source technologies.
Created AI-ML platform for automatic deployment of multiple Machine learnig applications on GPU using slurm/sca, docker, CICD tools, Oozie, Hadoop applications.
Created alerting and monitoring frameworks for batch & streaming jobs.
Created data validation framework to reinforce data quality, data governance checks.
Created data warehouse system for supply chain management.
Migrated data from multiple legacy systems (TeraData, CR and Informatica) to Hadoop using HDP.
Developed data ingestion, transformation and cleansing layer for unstructured, semi-structured data using Scala, Spark, Hive.
Performed optimisation and decreased resource utilisation of applications by 40%.
Decreased the usage of storage space by 35% by using various data compression techniques.
Implemented SparkSQL and Dataframes for faster testing and processing of data.
Parsed unstructured and semi-Structured data using PySpark.
Created various lambda functions for data cleansing and data transformation using python, Spark API.
Provided backend support by creating Tables, functions, procedures, indexing.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
