Senior GCP Data Engineer

Seeking a highly experienced GCP Data Engineer / Database Platform Engineer with strong expertise in Google Cloud Platform (GCP) Data Engineering Database Administration Kubernetes and Infrastructure Automation. The ideal candidate will have hands-on experience with Airflow Spark Composer Dataproc Dataflow Cloud SQL Spanner BigQuery and enterprise-scale database platforms. The role requires deep knowledge of MySQL PostgreSQL OLTP/OLAP architectures containerization cloud networking IAM and Infrastructure as Code (Terraform).

This position involves designing optimizing automating and supporting large-scale GCP data and database environments while driving database modernization cloud-native solutions troubleshooting and production support.

Mandatory Requirements

• Strong hands-on experience with:
  • Apache Airflow
  • Apache Spark
  • GCP Composer
  • Dataproc
  • Dataflow
• Expertise in GCP database services:
  • Cloud SQL
  • Cloud Spanner
  • BigQuery
• Deep expertise in:
  • MySQL
  • PostgreSQL
  • Database Administration (DBA)
  • Query Optimization
  • Schema Design
  • Performance Tuning
• Strong understanding of:
  • OLTP Architectures
  • OLAP Architectures
• Hands-on experience with:
  • Docker
  • Kubernetes
  • Helm
  • GKE/OpenShift/EKS/AKS
• Automation and scripting:
  • Python
  • Shell Scripting
  • Terraform
• GCP Networking and Security
• GCP IAM Administration
• Enterprise-level Production Support Experience

Key Responsibilities

• Design develop and maintain cloud-native data engineering solutions on GCP.
• Build and optimize data pipelines using Airflow Spark Dataproc and Dataflow.
• Manage and support GCP database platforms including Cloud SQL Spanner and BigQuery.
• Perform database performance tuning troubleshooting and query optimization.
• Design scalable OLTP and OLAP data architectures.
• Automate cloud infrastructure deployment using Terraform and scripting tools.
• Administer Kubernetes environments and containerized applications.
• Configure and maintain GCP IAM roles policies and service accounts.
• Support enterprise-scale production environments and incident management.
• Evaluate new database technologies through POCs and drive production adoption.
• Collaborate with application teams to resolve database connectivity and performance issues.

Core Technical Skills

Google Cloud Platform (GCP)

• Cloud SQL
• Cloud Spanner
• BigQuery
• Dataproc
• Dataflow
• Composer
• GKE
• Private Service Connect
• IAM
• Networking
• Cloud Operations

Data Engineering

• Apache Airflow
• Apache Spark
• ETL Pipelines
• Batch Processing
• Data Lake Architecture
• Data Warehousing

Databases

• MySQL
• PostgreSQL
• Database Administration
• Query Optimization
• Performance Tuning
• High Availability
• Backup & Recovery
• Replication

Containers & DevOps

• Docker
• Kubernetes
• Helm
• OpenShift
• AKS
• EKS
• GKE

Automation & Infrastructure

• Terraform
• Python
• Shell Scripting
• Infrastructure as Code (IaC)
• CI/CD

Nice to Have

• Java
• Spring Framework
• Hibernate
• JPA
• Connection Pooling
• Application Performance Tuning