Senior Data Engineer HFT

Hong Kong - Hong Kong

Monthly Salary: Not Disclosed

Posted on: Yesterday

Vacancies: 1 Vacancy

Job Summary

My client is seeking a Senior Data Engineer to join their dynamic team. This role is crucial for enhancing their data infrastructure and analytics capabilities in a fast-paced and rapidly evolving environment. The ideal candidate will have 4 years of hands-on experience and a strong command of data engineering principles and technologies.

Key Responsibilities

Design develop and maintain robust data pipelines for real-time analytics and machine learning applications.
Implement and optimize stream processing solutions using technologies such as Apache Flink or Spark.
Work collaboratively with quants data scientists and analysts to transform and validate datasets.
Ensure efficient data storage and retrieval using AWS services (S3 ClickHouse) and SQL solutions.

Core Technical Skills

Stream Processing: Expertise in Flink Spark Streaming PyFlink Kafka and AWS MSK.
Programming: Proficient in Python (including libraries like Pandas Polars boto3 and pyarrow) and SQL (Flink/Spark SQL).
Data Storage: Experience with S3 ClickHouse Parquet and JSON.
Monitoring/Logging: Familiarity with Grafana for data visualization and Fluentd for logging.

Cloud & Infrastructure

Proficient in AWS services such as EMR and Kinesis Data Analytics.
Strong knowledge of containerization technologies (Docker Kubernetes) and orchestration frameworks (Airflow YARN).
Familiarity with Infrastructure as Code (IaC) tools such as Terraform and Ansible is a plus.
Understanding CI/CD practices for DevOps integration.

Qualifications

Bachelors degree in Computer Science Engineering or related field (Masters preferred).
4 years of experience as a data engineer or in a similar role.
Strong problem-solving skills and ability to work under pressure in a fast-paced environment.