Data Platform Engineer

Core Competencies
- 5 years experience in Platform Engineering Infrastructure Engineering or similar roles supporting large-scale production environments.
- Strong hands-on experience administering Cloudera and distributed data processing platforms.
- Deep operational experience with Kafka cluster design administration and troubleshooting.
- Proven experience administering enterprise data governance and data quality platforms such as Collibra and Ataccama.
- Strong working knowledge of cloud platforms with hands-on Azure experience including Azure Synapse.
- Advanced experience with Infrastructure as Code tools including Terraform and Ansible.
- Strong proficiency in Bash and Python scripting.
- Expert-level understanding of Spark and PySpark including performance tuning and debugging.
- Strong experience operating containerised platforms using Kubernetes.
- Advanced SQL skills with proven experience tuning Hive queries.
- Hands-on experience building and operating CI/CD pipelines using Jenkins and GitHub.
- Strong experience administering PostgreSQL including replication backups and point-in-time recovery.
- Excellent troubleshooting analytical and problem-solving skills.
- Strong communication skills with the ability to influence technical and non-technical stakeholders.

Specific activities required
Lead the design and implementation of scalable resilient and secure platform architectures supporting enterprise data and application workloads.
Provide senior-level administration and operational oversight of Data Platforms including upgrades patching configuration and performance optimisation.
Own the reliability and performance of Kafka clusters ensuring high availability throughput and fault tolerance.
Provide technical leadership for data governance and data quality platforms including Collibra and Ataccama.
Design and support cloud-integrated data platforms with hands-on expertise in Azure services including Azure Synapse.
Define and implement Infrastructure as Code standards using Terraform and Ansible across platform environments.
Lead the design and implementation of CI/CD pipelines using Jenkins and GitHub to automate build deployment and configuration processes.
Architect and operate containerised platforms using Kubernetes including capacity planning and platform hardening.
Lead advanced troubleshooting of Spark and PySpark workloads including deep performance and stability analysis.
Drive SQL and Hive query performance tuning initiatives to improve platform efficiency and cost optimisation.
Oversee PostgreSQL administration including replication point-in-time recovery backup strategies and availability design.
Establish platform monitoring alerting and observability standards using logs metrics and performance indicators.
Act as the senior escalation point for complex production incidents and lead root cause analysis and remediation efforts.
Mentor and guide junior platform engineers promoting best practices and technical excellence.
Collaborate with architecture security data and application teams to align platform solutions with enterprise standards.
Own and maintain platform documentation standards and operational runbooks.
Participate in and lead on-call production support through a rotation schedule.