| Data Pipeline Development: Build and maintain scalable data pipelines using GCP services like Dataflow Composer and Dataproc. Automate data pipeline deployment to reduce manual interventions and increase reliability. Continuously monitor and optimize pipelines for performance scalability and cost-efficiency. Utilize Kafka or Pub/Sub for real-time data streaming and processing. Data Integration and Processing: Integrate data from various sources including relational databases (e.g. PostgreSQL SQL Server) and NoSQL databases (e.g. BigTable MongoDB Cosmos DB HBase). Develop ETL/ELT processes to load data into warehouses and storage systems while ensuring data quality and integrity. Leverage Python and Functional Programming principals to build efficient data integration solutions. Data Warehouse Optimization: Design and maintain efficient accessible and optimized data warehouses using tools like BigQuery. Improve performance through indexing partitioning and query optimization. SQL Optimization and Support: Write and optimize complex SQL queries including querying XML and JSON data structures. Provide training and support on advanced SQL techniques and best practices. Cloud-Native Solutions: Collaborate with architects to design and implement secure cloud-native solutions. Implement data security best practices including encryption access controls and compliance with regulatory standards (e.g. GDPR HIPAA). Work with Kubernetes and containerization tools to deploy and scale cloud-native applications. Version Control and CI/CD Pipelines: Use tools like GitHub for version control to track changes. | 
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
