Job Summary:
We are seeking a talented Data Engineer with strong experience in FastAPI and a solid background in Google Cloud Platform (GCP) services. This role involves building scalable data pipelines developing APIs and leveraging GCP tools to support data infrastructure and analytics.
Key Responsibilities:
- Design and implement scalable data pipelines using GCP-native tools.
- Develop and maintain RESTful APIs using FastAPI for data access and integration.
- Work with large-scale datasets from various sources (structured and unstructured).
- Optimize data workflows for performance scalability and reliability.
- Collaborate with cross-functional teams including data scientists analysts and backend engineers.
- Ensure data quality integrity and security across all pipelines and systems.
Required Skills:
- Strong experience with FastAPI and general API development.
- Proficiency in Python and SQL.
- Hands-on experience with GCP services such as:
- BigQuery
- Cloud Storage
- Cloud Functions
- Pub/Sub
- Dataflow
- Composer (Airflow on GCP)
- Familiarity with CI/CD pipelines and version control (Git).
Preferred Qualifications:
- Experience with containerization (Docker Kubernetes).
- Understanding of data governance and security best practices.
- Exposure to real-time data processing and streaming architectures.