Hardworking and reliable Data Engineer with strong ability in designing pipeline and analytics with 2.5 years of experience.
GCP Data Engineer | I have experience in designing pipelines and monitoring with GCP services like Bigquery, pubsub, cloud storage, and cloud function. I have worked on code enhancement and impact analysis work as well. I have implemented automation scripts in Python.
<p>Designed and implemented a robust data pipeline utilizing Apache Airflow and Google Cloud Dataflow, seamlessly ingesting Pub/Sub messages into BigQuery tables, enhancing real-time data processing and storage efficiency.<br>
Engineered a comprehensive solution for transforming BigQuery tables as per ML model results and publishing details to Pub/Sub topics, demonstrating proficiency in end-to-end data processing, orchestration, and message communication.<br>
Demonstrated expertise in GCP's data ecosystem by orchestrating the entire workflow, ensuring smooth integration of Pub/Sub, BigQuery, and Dataflow.<br>
Sound knowledge on implementing pipeline with Apache Beam Python SDK.</p>
<p>Collaborated with the development team to write Hive queries and its enhancement on Hadoop with YARN cluster manager.<br>
Worked on on-premises to GCP migration and conducted root cause analysis for failing queries, ensuring seamless data processing and analysis.<br>
Utilized PySpark scripting to automate the creation of 100+ source tables, increasing 50% speed of QA and DEV initial efforts.<br>
Implemented a PySpark script to automate HDFS Commands and Hive query to perform comprehensive verification of 1:1 and transformation logic with report generation saved time required for quality check.<br>
Developed a Python Machine Learning - Time Series model (ARIMA) to accurately predict the duration of job execution, resulting in improved efficiency and optimized resource allocation.<br>
Implemented a robust script that parses JIRA exports and calculates defect counts for each entity, reducing manual effort and ensuring quick and accurate defect analysis.</p>
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
