Tech stack/ knowledge:
- Mandatory:
- Proficient in relational databases and SQL.
- Proven big data experience either from an implementation or a data science perspective.
- Strong coding experience in the likes of Java Scala or Python.
- Experience with GCP tech stack (BigQuery Dataflow Cloud Composer Google Cloud Storage) or other public cloud providers equivalents (AWS Azure)
- Experience with Git Maven CI/CD (eg. Azure DevOps) and documentation tools
- Experience with Infrastructure as Code (eg. Terraform)
- Experience or affinity withIT Securityconcepts
- Nice to have:
- DBT
- Looker
- Data governance Data lineage Data Catalogue (Metadata)
- Apache Beam
- Spark
- Flink
- Kafka
- Automated testing framework
Foreign languages: English (advanced)
Here are your daily responsibilities:
- As a Senior Data Engineer you contribute and are responsible for the design development test automation hardening (security stability deployment) and documentation of components and data pipelines for the extraction and provision of structured and unstructured data within a cloud infrastructure or between cloud and onpremises systems.
- You contribute totechnology choicesandarchitectureof newly built components and services
- You have a strong focus onperformance large data sets and towardseventdrivenarchitecture
- You assess the efficiency of endtoend processes by identifying and mitigating risks
- You continuously improve and optimize the ingestion postprocessing and reporting layer of the data platform
- You standardize and reutilize automation patterns as part of the data ingestion and processing stages
- You designanddevelopautomatedunit integration and regression tests
- You prove strong analytical skills proactivity and ability to work in crossborder international teams.
- You work in an Agile environment collaborate closely with the Product Owner and Customer Journey Expert to interpret business requirements and estimate plan implement test and deliver as part of short delivery cycles.
- You focus on deliveringworking softwareofhigh qualitythat satisfies customer requirements.