About Me
Data Engineer with a strong drive for growth and career advancement in the dynamic field of data engineering.
Experience
Data Engineer
Collecting E-retail data to enhances Datagram platform’s Analytics.
Enhanced Spark based ETL jobs, boosting efficiency and ensuring seamless migration across PostgreSQL and Cassandra databases.
Led the implementation of automation utilizing Kafka and Python, resulting in a significant reduction in data processing time.
Implemented robust data quality validation with Great Expectations, ensuring high accuracy across large datasets.
Developed a Monitoring Dashboard for comprehensive data oversight using Apache Superset.
Data Engineer
• Enhanced Spark based ETL jobs, boosting efficiency and ensuring
seamless migration across PostgreSQL and Cassandra databases.
• Led the implementation of automation utilizing Kafka and Python,
resulting in a significant reduction in data processing time.
• Implemented robust data quality validation with Great Expectations,
ensuring high accuracy across large datasets.
• Developed a Monitoring Dashboard for comprehensive data
oversight using Apache Superset.
Data Engineer / scientist
Combine raw data and Prepare database for modeling water supply network through GIS tools.
Daily drinking water demand forecasting, artificial neural network.
water leak detection using machine learning algorithms.
Data scientist intern
Extract the visual attributes using the Resnet-152 network with an attention mechanism.
Extract the textual attributes using GRUs similar to RNN using a pre-trained Skip-Thoughts model recognition for collecting product data from web pages.
Training and validation of the multimodal model, and Classification using the KNN.