Middle Data Engineer (GCPBigQuery Migration)
Share
Job Location:
Monthly Salary:
Posted on:
5 hours ago
Vacancies:
1 Vacancy
Department:
Job Summary
In this role you will help build and maintain data pipelines implement CDC processes and support the migration from RDS to BigQuery. Youll work independently while receiving direction from senior engineers and the Engineering Manager.
Working hours: 14:0023:00
Responsibilities
- Lead the migration of data from RDS to Google BigQuery.
- Design construct test and maintain highly scalable data management systems.
- Ensure systems meet business requirements and industry practices.
- Integrate new data management technologies and software engineering tools into existing structures.
- Work with messaging systems and data pipelines.
- Employ a variety of technologies and tools to marry systems together.
- Drive the improvements on data reliability efficiency and quality.
Qualifications :
- 5 years of hands-on experience with Google Cloud Platform including: BigQuery DataStream (CDC) Dataflow Pub/Sub
- Strong Java development background.
- Experience with Spring Boot.
- Proven expertise in building CDC pipelines and distributed data processing systems.
- Track record of migrating large datasets from relational databases into BigQuery.
- Strong knowledge of automated testing (unit integration pipeline tests).
- Ability to work autonomously make architectural decisions and own end-to-end delivery.
Remote Work :
Yes
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small player ... View more
