GCP Data Engineer

Nashville, IN - USA

Monthly Salary: Not Disclosed

Posted on: 8 hours ago

Vacancies: 1 Vacancy

Job Summary

Key Responsibilities:

Architect Scalable Data Solutions: Design and implement data warehouses marts lakes and batch and/or real-time streaming pipelines using GCP-native tools.
Data Modeling & Integration: Design and Develop conformed data models (star/snowflake schemas) and ETL/ELT processes for analytics and BI tools (MicroStrategy Looker Power BI).
Pipeline Development: Build scalable pipelines and automate data ingestion and transformation workflows using BigQuery Dataflow Dataproc/PySpark Cloud Functions Pub/Sub Kafka and Cloud Composer for orchestration.
Security & Compliance: Implement IAM encryption and compliance standards (GDPR HIPAA) with GCP security tools.
Performance Optimization: Apply best practices for partitioning clustering and BI Engine to ensure high performance and cost efficiency.
DevOps & Automation: Integrate CI/CD pipelines IaC (Terraform) and containerization (Docker Kubernetes) for deployment and scalability.
Collaboration & Leadership: Engage with stakeholders including leadership Project Managers BAs Engineers QA platform teams mentor teams and provide technical guidance on best practices.
Troubleshooting: Resolve complex technical issues and support incident response.
Healthcare Domain Expertise: Ensure compliance with healthcare regulations and stay updated on industry trends.

Required Skills & Working experience:

GCP Expertise: BigQuery Cloud Storage Dataflow (Apache Beam with python) Dataproc/PySpark Cloud Functions Pub/Sub Kafka Cloud Composer.
Programming: Advanced SQL and Python for analytics and pipeline development.
Performance Optimization: Experience with optimization of query performance partitioning clustering and BI Engine in BigQuery.
Automation: Experience with CI/CD for data pipelines IaC for data services automation of ETL/ELT processes.
Security: Strong knowledge of IAM encryption and compliance frameworks.
Architecture Design: Ability to create fault-tolerant highly available and cost-optimized solutions.
Communication: Excellent ability to convey technical concepts to both technical and non-technical stakeholders.
Domain Knowledge: Familiarity with healthcare data management and regulatory compliance.

Key Responsibilities: Architect Scalable Data Solutions: Design and implement data warehouses marts lakes and batch and/or real-time streaming pipelines using GCP-native tools. Data Modeling & Integration: Design and Develop conformed data models (star/snowflake schemas) and ETL/ELT processes f...

Key Responsibilities:

Architect Scalable Data Solutions: Design and implement data warehouses marts lakes and batch and/or real-time streaming pipelines using GCP-native tools.
Data Modeling & Integration: Design and Develop conformed data models (star/snowflake schemas) and ETL/ELT processes for analytics and BI tools (MicroStrategy Looker Power BI).
Pipeline Development: Build scalable pipelines and automate data ingestion and transformation workflows using BigQuery Dataflow Dataproc/PySpark Cloud Functions Pub/Sub Kafka and Cloud Composer for orchestration.
Security & Compliance: Implement IAM encryption and compliance standards (GDPR HIPAA) with GCP security tools.
Performance Optimization: Apply best practices for partitioning clustering and BI Engine to ensure high performance and cost efficiency.
DevOps & Automation: Integrate CI/CD pipelines IaC (Terraform) and containerization (Docker Kubernetes) for deployment and scalability.
Collaboration & Leadership: Engage with stakeholders including leadership Project Managers BAs Engineers QA platform teams mentor teams and provide technical guidance on best practices.
Troubleshooting: Resolve complex technical issues and support incident response.
Healthcare Domain Expertise: Ensure compliance with healthcare regulations and stay updated on industry trends.

Required Skills & Working experience:

GCP Expertise: BigQuery Cloud Storage Dataflow (Apache Beam with python) Dataproc/PySpark Cloud Functions Pub/Sub Kafka Cloud Composer.
Programming: Advanced SQL and Python for analytics and pipeline development.
Performance Optimization: Experience with optimization of query performance partitioning clustering and BI Engine in BigQuery.
Automation: Experience with CI/CD for data pipelines IaC for data services automation of ETL/ELT processes.
Security: Strong knowledge of IAM encryption and compliance frameworks.
Architecture Design: Ability to create fault-tolerant highly available and cost-optimized solutions.
Communication: Excellent ability to convey technical concepts to both technical and non-technical stakeholders.
Domain Knowledge: Familiarity with healthcare data management and regulatory compliance.