In this role youll work cross-functionally across product and data science teams to build large-scale batch and streaming data pipelines that power Analytics Experimentation and Machine Learning. Youll design instrumentation on both client and server sides to ensure accurate and reliable data collection. Youll validate that data flows with the right structure frequency and quality. Youll build clean high-performance data models and develop self-serve tools that make data easier to consume and scale. Youll automate dataset lifecycles with strong quality standards and help partners confidently use the data for product insights.
7 years of experience designing building and maintaining distributed data processing systems at scale.
5 years of hands-on experience with batch and/or streaming data technologies such as Spark Flink Kafka Presto Hadoop and Iceberg.
Strong data modeling and SQL skills and experience working with large-scale complex and high-dimensional datasets.
Proficient in at least one modern programming language (e.g. Python Java and Scala).
MS or BS in Computer Science Engineering Math Statistics or a related field or equivalent practical experience in data engineering.
Experience with machine learning algorithms or pipelines particularly in the context of data engineering.
Experience supporting ML engineers or data scientists with feature engineering or model data pipelines is a plus.
Familiarity with testing tools and methodologies for validating large-scale distributed data systems (e.g. data quality checks pipeline testing frameworks fault tolerance testing).
Proven software engineering fundamentals including experience with design testing version control and CI/CD best practices.
Comfortable working independently in a fast-paced ambiguous environment.
Excellent communication and problem-solving skills.
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.