GCP Data Engineer (AI Experience)

Job Description:

• Required - Must have experience AI - GPT Gemini. Must be able to demonstrate how they use it. Using it as business use cases on resume. Must be highlighted clear on the resume
• Required - minimum 7 years of proven hands-on experience with GCP data services.
• Preferred - GCP Google Cloud Professional Data Engineer Certification.
• Required Strong understanding of distributed systems and data engineering best practices.
• Required Excellent communication and documentation skills.
• Required Ability to work independently and collaboratively in a fast-paced environment.
• Required experience in Agile methodology with preferred experience in SAFe methodology
• Required good communication and collaborative skills
• Required experience working in onshore/offshore support model collaborating work with offshore teams.
• Preferred experience working with GitHub RTC automation tools.
• Required - minimum 3 years of proven hands-on experience in the following:

Responsibilities

• Design and implement robust data pipelines using Google Cloud Platform (GCP) services such as BigQuery Cloud Storage and Pub/Sub.
• Develop and manage workflows using Cloud Composer (Apache Airflow) for efficient scheduling and orchestration.
• Write clean efficient and scalable code in Python leveraging advanced programming techniques.
• Craft complex SQL queries in BigQuery including window functions CTEs and performance tuning strategies.
• Build and maintain real-time data processing systems using Apache Kafka.
• Model and manage NoSQL databases particularly MongoDB with a focus on scalable schema design.
• Utilize Shell scripting and perform Linux system administration tasks to support data infrastructure.
• Conduct data profiling and implement validation techniques to ensure data quality and integrity.
• Develop and maintain API integration scripts for seamless service automation and data exchange
• Troubleshoot and resolve data-related issues with strong analytical and problem-solving skills.
• Create and maintain data flow diagrams to clearly communicate architecture and pipeline logic to stakeholders.
• Develop and manage workflows using Cloud Composer 3