Data Engineer

Hong Kong - Hong Kong

Monthly Salary: Not Disclosed

Posted on: 4 hours ago

Vacancies: 1 Vacancy

Job Summary

About the role:

You are pioneering and innovative and want to be part of the cutting-edge and disruptive crypto-currency world
You are eager to learn new knowledge in both financial and technical fields
You thrive in a non-hierarchical organization with a casual working environment
You enjoy solving complex distributed systems challenges and optimizing streaming data pipelines
You value comprehensive documentation and collaborative problem-solving

As a Data Engineer you will:

Build real-time data pipelines using Apache Flink (PyFlink) to process high-volume logs through EC2 Vector MSK PyFlink S3 ClickHouse architecture
Design stream processing systems with watermark strategies window operations exactly-once semantics and state management for critical data
Deploy and manage AWS infrastructure including Managed Flink S3 data lakes MSK IAM and CloudWatch monitoring with optimized partitioning strategies
Optimize performance and troubleshoot SQL queries (Flink/ClickHouse) production issues data skew and build Grafana dashboards for pipeline monitoring
Implement data quality frameworks with schema evolution validation strategies and translate business requirements into scalable data solutions
Manage DevOps processes including JAR dependencies Docker containerization Kubernetes deployments and comprehensive documentation

Qualifications:

University degree in Computer Science Software Engineering or related disciplines
Apache Flink/PyFlink with watermarks state management and window operations
Python (pandas polars boto3) and SQL (Flink SQL ClickHouse)
Kafka/AWS MSK and message streaming concepts
AWS services: Managed Flink S3 MSK CloudWatch IAM
Docker Kubernetes and containerization fundamentals
Grafana monitoring and production troubleshooting experience

About the role: You are pioneering and innovative and want to be part of the cutting-edge and disruptive crypto-currency worldYou are eager to learn new knowledge in both financial and technical fieldsYou thrive in a non-hierarchical organization with a casual working environmentYou enjoy solving co...