Back-end developer
Back-end developer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
At Heuritech we develop crawlers for various data sources (mainly social
media i.e Instagram Weibo TikTok) to gather millions of lines of data (posts
authors images) on a recurring basis. This data then goes through our data
pipeline it get analyzed by our computer vision modules these predictions
then get aggregated into time series that are used to build ongoing and
forecasted metrics in order to feed our product.
Data Engineering Team
The Data Engineering team is responsible for building maintaining and
monitoring the data pipeline that processes millions of posts from thousands of
authors from their crawling on social media to their analysis by computer
vision modules and their aggregation into time series that are used to build
relevant metrics and forecasts to feed out product.
The data we crawl is inserted into our data warehouse and transformed along
the way to lead to relevant metrics that can either feed our product or be
accessible through our API(s).
Your Role
As part of the Data Engineering team your role will be to build and maintain
robust and scalable components for the data pipeline from the gathering of
online content to their processing and transformations that lead to product
insights. This includes the following tasks:
Develop crawlers for new data sources and integrate them into data pipelines
Maintain the current crawling codebase
Expand our geographical and segmented coverage
Monitor data flows through relevant metrics
Optimize data processing and transformations
Develop tools usable by nontech teams to access information regarding the crawled data
Employment Type
Full-Time
