Web Scrape Engineer

SO New Requirement

Web Scrape Engineer

Job Location: Halifax (Candidates needs to work from Day 1 Onsite 3 Days at client Office)

As a Web Scraping focused Data Engineer you will be responsible for extracting and ingesting data from websites using web crawling tools. In this role you will own the creation process of these tools services and workflows to improve crawl/ scrape analysis reports and data management.

• Experience running large scale of web scrapes.
• Experience in analyzing web scraping requirements.
• Communicating with third party vendors on specific data requirements for web scraping
• Develop custom scripts and workflows using Python SQL and C# to automate data processing tasks.
• Familiarity with techniques and tools for crawling extracting and processing data (e.g. Scrapy pandas MapReduce SQL BeautifulSoup etc.).
• Strong grasp of data modeling concepts to design and develop efficient data storage and retrieval systems.
• Minimum 4 years of experience as a Data Engineering with a Masters degree; or 5 years with a Bachelors degree along with the relevant working experience.
• 23 years Financial Industry experience.
• Experience working as a Data Engineer in a production environment.
• Experience working with a modern scalable Data Lake or Data warehouses in Snowflake.
• 5 years of proficient experience working with programming languages such as Python PySpark SQL Scala Shell scripting etc.
• Experience understanding of the Spark Architecture is preferred.
• Preferred one or more Database experience (MySQL Microsoft SQL Server MongoDB PostgreSQL)
• Experience working with containers and orchestration tools like (Docker Kubernetes Apache Airflow CI/CD etc.)
• Experience in promoting data ingestion pipelines by using CI/CD e.g. Jenkins.
• Excellent written and verbal communication presentation skills.
• Experience working with one or more cloud platforms (Azure AWS or GCP )
• preferred: Azure
• Experience working with distributed notebook environments like Databricks Azure Synapse etc.
• Experience working with Git Azure DevOps.
• Understanding of Machine learning algorithms i.e. Anomaly detection
• Ability to work in Agile methodology.
• Transforming and manipulating raw complex data into structured and consumable format data
• Machine Learning and Quantitative Modeling
• Build anomaly detection model leveraging packages like Prophet or similar
• Build anomaly detection models for geospatial and other practices based on domain requirements.

Required

• Selfstarter and delivery oriented experienced Machine Learning engineer
• Experience working with alternative data
• Experience with Cloud Distributed computing and Data Science
• Excellent communication and presentation skills. Proven ability to connect with internal and external stakeholders and excel in a fastpaced environment

Impact

• Revenue generation thru New Business for Alternative Data
• Innovation
• 6 years of AI Big Data and cloud expertise
• 34 years of Alternative data experience

Risk

• Mitigate reputation risk thru AI driven Data Quality to ensure highest quality data and services are offered to clients