Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Data Engineer
Location: Remote
Duration: Contract
Job Summary
Must complete the Data Engineer Glider
Job Description
As a Data Engineer you will be responsible for designing and implementing a cloud-native data processing and API integration system. This role involves ingesting identity data from upstream sources detecting record-level changes and synchronizing user metadata to a downstream system via API. This is a position that focuses on scalable data handling automation and fault-tolerant service deployment within Google Cloud Platform (GCP).
What Youll Do
- Solution Design & Development: Build modular Python applications that process identity data files or APIs and sync them to target platforms
- Data Staging & Processing: Stage identity metadata in BigQuery using defined schemas and implement change detection logic (create/update/delete)
- API Integration: Design and implement logic to call RESTful APIs to maintain target user repositories (e.g. user attributes roles)
- Workflow Orchestration: Use GCP Pub/Sub Composer and/or Cloud Run to manage asynchronous workflows and ensure event-driven processing
- Infrastructure as Code: Deploy and manage services using Terraform with a focus on security idempotency and configuration as code
- Observability & Resilience: Implement logging retry logic and incident handling to ensure system reliability and traceability
- Testing & Validation: Build automated test coverage for critical processing logic and API interactions
What Youll Need
Required:
- Bachelors or masters degree in computer science data engineering or equivalent work experience
- 6 years in backend development or data engineering roles focused on identity security or metadata systems
- Strong Python engineering for data processing and backend development
- Advanced experience with GCP services: BigQuery Cloud Run Cloud Functions Cloud Composer Pub/Sub Cloud Storage Secret Manager Cloud Scheduler
- Interacting with REST APIs including OAuth2 or token-based authentication
- Terraform for cloud infrastructure automation
- Proficiency with SQL for data transformation and validation
- Strong understanding of CI/CD containers (Docker) Git workflows
- Comfortable working with structured metadata user roles and directory-style data
- Able to work independently and meet delivery milestones
- Strong documentation and debugging skills
- Must adhere to enterprise security and change control practices
Preferred:
- Experience integrating with IAM or identity systems (e.g. LDAP Okta custom directories)
- Background working in regulated or high-security environments
- Experience handling large-scale user datasets (millions of records)
- Familiarity with hybrid data processing (batch streaming)
- GCP Certifications
Physical Demands:
- Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal state and local standards
- Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings etc.
- Ability to conduct repetitive tasks on a computer utilizing a mouse keyboard and monitor
Glider,Python
Employment Type
Full Time
