Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Location of Job: Remote- Canada
We are a dynamic team dedicated to solving complex problems in the realm of online retail product data. Our mission is to collect categorize and analyze vast amounts of semi-structured data from online retailers across the US and Canada. We empower our customers to gain a competitive edge by uncovering valuable insights hidden within this data.
We are seeking a talented Senior Data Engineer to join our Matching team. As a key member of our team you will play a pivotal role in developing the next-generation consolidated Matching platform capable of handling over 200 million transactions per day. Your expertise will be instrumental in designing building and optimizing high-performance scalable solutions that enable our customers to extract maximum value from their data.
If you are a highly motivated and talented software engineer who thrives in a fast-paced environment we encourage you to apply. Join our team and make a significant impact on the future of product data analysis.
Essential Functions
Data Integration and Matching: Design and implement robust data integration pipelines to collect clean and standardize product data from various sources. Develop advanced matching algorithms to identify and correlate products across different retailers.
Platform Software Development: Research evaluate and synthesize users requirements and other technical information to design and develop our distributed software platform ensuring its scalability reliability and performance. Collaborate with other engineers to architect and implement microservices-based architectures.
Algorithm Optimization: Continuously optimize matching algorithms and data processing pipelines to improve efficiency and accuracy. Leverage your expertise in data structures algorithms and performance tuning to enhance system responsiveness.
Technical Innovation: Apply the latest technologies to enhance software performance and security. Troubleshoot and resolve complex technical issues. Translate business needs into efficient technical solutions. Drive improvements in software development processes and promote continuous learning.
Technical Leadership: Foster a collaborative team environment. Mentor and guide junior developers fostering a culture of innovation and excellence. Share your knowledge and experience to drive best practices and ensure high-quality code. Coordinate between technical teams and stakeholders in the development of software and integrated information systems.
Problem-Solving: Collaborate with cross-functional teams to identify and solve complex technical challenges. Apply your analytical skills and problem-solving abilities to deliver innovative solutions.
Technologies We Use
Languages: Python SQL NodeJS/NestJS/Typescript React/Redux
Infrastructure: AWS Docker Kubernetes Apache Spark Apache Airflow Terraform GitHub Actions ArgoCD
Databases: MongoDB Postgres MySQL Trino Iceberg Redis Memcached
Streaming and Queuing: Kafka NATS RabbitMQ Keda
Qualifications :
Bachelors/Masters degree in Computer Science or relevant technical degree
Over 7 years of professional experience in software and data engineering.
Strong proficiency in Python and with a deep understanding of distributed systems and cloud-native technologies
Strong proficiency working of RDMS/NoSQL/Big Data solutions (Postgres MongoDB Trino etc.)
Proficiency working with data processing technologies such as Spark Flink and Airflow
Solid understanding of Data Lakes Data Warehouses and modeling practices (Data Vault etc.)
Understanding of streaming solutions such as Kafka Pulsar Kinesis/Firehose etc.
Excellent understanding of Docker and Kubernetes
Solid understanding of ETL/ELT and OLTP/OLAP concepts
Solid understanding of columnar/row-oriented data structures (e.g. Parquet ORC Avro etc.)
Mastery of AWS Linux and infrastructure as code.
Experience leveraging data lake solutions (e.g. AWS Glue DBT Trino Iceberg etc.)
Understanding of Domain Driven Design concepts and Microservices Architecture
Experience working in an agile team environment
Bonus Points
Experience working within a retail or ecommerce industry.
Full stack engineering experience including front end frameworks (React Redux)
Experience with CQRS
Experience with C4 modeling
Additional Information :
Other Duties - Please note this job description is not designed to cover or contain a comprehensive listing of activities duties or responsibilities that are required of the employee for this job. Duties responsibilities and activities may change at any time with or without notice.
EEO STATEMENT - Wiser Solutions Inc. is an Equal Opportunity Employer and prohibits Discrimination Harassment and Retaliation of any kind. Wiser Solutions Inc. is committed to the principle of equal employment opportunity for all employees and applicants providing a work environment free of discrimination harassment and retaliation. All employment decisions at Wiser Solutions Inc. are based on business needs job requirements and individual qualifications without regard to race color religion sex national origin family or parental status disability genetics age sexual orientation veteran status or any other status protected by the state federal or local law. Wiser Solutions Inc. will not tolerate discrimination harassment or retaliation based on any of these characteristics.
Base pay is one part of our total compensation package. Pay is established on an individual basis after considering multiple factors such as relevant experience education and other qualifications. In addition we take into account geographical differentials and make sure pay is equitable with our current staff. For this position our hiring range for base annual pay is estimated to be CAD$120000 to $140000 at the time of this posting.
Performance-based discretionary bonuses and variable pay plans are available for some positions.
If you require accommodation to complete any part of the application process or need an alternative manner to apply please contact us at or call .
Remote Work :
No
Employment Type :
Full-time
Employment Type
Full-time
