GCP Data Engineer

Randstad India

Job Location:

Bangalore - India

Monthly Salary: Not Disclosed

Posted on: 7 hours ago

Vacancies: 1 Vacancy

Job Summary

Key Responsibilities

Design and execute comprehensive data migration strategies from
onpremises legacy platforms or other cloud environments to GCP.
Migrate structured and semistructured data to BigQuery Bigtable and other
GCP storage solutions.
Build optimize and maintain scalable migration pipelines using Python SQL
and distributed processing frameworks on Dataproc (Apache Spark).
Implement batch and streaming migration patterns using Cloud Composer
(Apache Airflow) Kafka and Google Pub/Sub.
Perform schema mapping data transformation reconciliation and validation
to ensure accuracy consistency and completeness of migrated datasets.
Enable metadata management lineage and governance using Dataplex and
Data Catalog.
Handle largevolume data transfers using industry best practices such as
parallel loading incremental loads and CDC where applicable.
Lead migrations from onpremise RDBMS enterprise data warehouses or
Hadoop ecosystems.
Collaborate closely with application data infrastructure and security teams to
ensure seamless migrations with minimal business disruption.
Optimize migrated workloads for performance scalability and cost efficiency
on GCP.
Troubleshoot migration issues and support postmigration stabilization and
continuous optimization.
Document migration architectures processes and operational runbooks.

Required Technical Skills

Google Cloud Platform (GCP)
BigQuery
Dataproc / Apache Spark
Python (ETL/ELT pipelines and automation)
SQL (data transformation validation and reconciliation)
Databases: MySQL Hive MongoDB
Cloud Composer / Apache Airflow
Kafka and Google Pub/Sub
Bigtable
Apigee / REST APIs (data extraction and integration)
Dataplex / Data Catalog (metadata management and governance)

Good to Have
Experience with Change Data Capture (CDC) patterns
Data quality audit and reconciliation frameworks
CI/CD implementation for data pipelines

Key Responsibilities Design and execute comprehensive data migration strategies from onpremises legacy platforms or other cloud environments to GCP. Migrate structured and semistructured data to BigQuery Bigtable and other GCP storage solutions. Build optimize and maintain scalable migr...