Data Platform Engineer
Data Platform Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
ING Hubs Poland is hiring!
We are looking for you if you:
- have 5 years of experience in aPlatform Engineer role who has attained a Graduate degree in Computer Science Statistics Informatics Information Systems or another quantitative field
- have experience with advanced Azure Dev/Ops pipelines
- have experience using one or more of the following software components:
-The Cloudera Eco System (HDFS HIVE NIFI)
- Management of the bigdata clusters using CM Ranger KNOX and HUE
- Produce and Consume streaming data (KAFKA)
- Experience with other storage layers (e.g. S3 CEPH) - know development languages such as Python Ansible and Bash
- know how to build Cloud agnostic data capabilities
- have knowledge about public Cloud (GCP Azure)
- know monitoring tools like Grafana Kibana Prometheus and Elasticsearch.
- have experience building and optimizing big data data pipelines architectures and data both batch and real-time data integration
- have knowledge of message queuing and stream processing
- have worked in a DevOps environment: CI/CD with Azure DevOps Test automation Docker
- Agile Scrum way of working.
Youll get extra points for:
- excellent communication skills both verbally and in writing to be able to align with various stakeholdersand cross border squads
- results oriented determined and having a self-driven attitude.
Your responsibilities:
- define and implement the infrastructure required for optimal loading of data from a wide variety of data sources using big data technologies
- operate and maintain a critical environment
- setup and tune the monitoring of the infrastructure including data pipelines
- identify design and implement internal process improvements: automating manual processes optimizing data delivery re-designing infrastructure for greater scalability etc
- work with stakeholders including the Data and Design teams to assist with data-related technical issues and support their data infrastructure needs
- keep our data separated and secure across international boundaries through multiple data centers.
About the team:
Have you ever build the best Data Platform ever
If your answer is yes then you are the Data Platform engineer we are looking for.
You can help us tobuild expand and maintain this together with ateam of know about batch and streaming ingestion and that security and data management in a data environment are must haves. You will work in a team where you learn from others and share your knowledge have fun and make ING data driven.
Our ideal candidate is an experiencedengineer managing large data environments based on the Cloudera Data Platform (Hadoop) Cloudera Flow Manager (CFM) and other storage layer technologies.
The Platform Engineer will actively support our software developerson data initiatives and will ensure optimal data delivery.
Experience in the Agile methodologies would be a perfect fit able to slice and dice work/ideas in clear stories and tasks.
The role naming convention in the global ING job architecture will be Engineer II.
Employment Type
Full-Time
