نبذة عني
Having 3 Years of IT experience in design and development using GCP Components like Dataflow, Big Query, Cloud SQL, Pub/Sub, Cloud Composer, Cloud storage and Compute Engine. Experience working with Big Query to load dat…
Having 3 Years of IT experience in design and development using GCP Components like Dataflow, Big Query, Cloud SQL, Pub/Sub, Cloud Composer, Cloud storage and Compute Engine. Experience working with Big Query to load data to Stage, History and Control tables, Developed Authorized views. Worked on Creating DAG’s and tasks to deploy and run PySpark jobs in the Airflow scheduling tool. Awareness in performance tuning, deployment procedures, troubleshooting and solving database issues. Experience on python language scripting. Good knowledge on Big Data technologies like Hadoop, and Apache Spark. Used GIT to manage and deploy the code version. Expertise in implementing end-to-end Cloud computing projects using standard SDLC. Good experience in all phases of Agile and participated in daily scrum meetings.
الخبرة
gcp data engineer
Developed BQ DDL and DML scripts to Load data into BigQuery from GCS buckets.
Analyze and organize the raw data as per the business requirements.
Creation of BigQuery Data sets, tables and pipelines to store the processed results.
Used cloud shell in GCP to configure the services Storage, BigQuery.
Experience in handling CSV, Avro, Parquet file formats.
Preparation of data for prescriptive and predictive modeling.
Developed Audit tables for reconciliation and metadata tracking purposes.
Create authorized views in BigQuery to provide required data to downstream applications
Developed BQ DDL and DML scripts to load data into BigQuery from GCS buckets.
Analyze and organize the raw data as per the business requirements.
Creation of BigQuery data sets, tables and pipelines to store the processed results.
Used Cloud Shell in GCP to configure the services Storage and BigQuery.
Experience in handling CSV, Avro and Parquet file formats.
Preparation of data for prescriptive and predictive modeling.
Developed audit tables for reconciliation and metadata tracking purposes.
Create authorized views in BigQuery to provide required data to downstream applications.
Using source code management as a GitHub tool.
Responsible for Unit Testing and SIT (System integration Testing) and capturing the evidence.
Responsible for SIT walk through to Business to get sign-off.
Chemical Inventory Management System
SQL, Big Query, Dataproc, Pub/Sub, Airflow/Composer