Lead Data Engineer
Lead Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
When looking to buy a product whether it is in a brick and mortar store or online it can be hard enough to find one that not only has the characteristics you are looking for but is also at a price that you are willing to pay. It can also be especially frustrating when you finally find one but it is out of stock. Likewise brands and retailers can have a difficult time getting the visibility they need to ensure you have the most seamless experience as possible in selecting their product. We at Wiser believe that shoppers should have this seamless experience and we want to do that by providing the brands and retailers the visibility they need to make that belief a reality.
Our goal is to solve a messy problem elegantly and cost effectively. Our job is to collect categorize and analyze lots of structured and semistructured data from lots of different places every day (whether its 20 million products from 500 websites or data collected from over 300000 brick and mortar stores across the country). We help our customers be more competitive by discovering interesting patterns in this data they can use to their advantage while being uniquely positioned to be able to do this across both online and instore.
We are looking for a leadlevel software engineer to lead the charge on a team of likeminded individuals responsible for developing the data architecture that powers our data collection process and analytics platform. If you have a passion for optimization scaling and integration challenges this may be the role for you.
What You Will Do
- Think like our customers you will work with product and engineering leaders to define data solutions that support customers business practices.
- Design/develop/extend our data pipeline services and architecture to implement your solutions you will be collaborating on some of the most important and complex parts of our system that form the foundation for the business value our organization provides
- Foster team growth provide mentorship to both junior team members and evangelizing expertise to those on others.
- Improve the quality of our solutions help to build enduring trust within our organization and amongst our customers by ensuring high quality standards of the data we manage
- Own your work you will take responsibility to shepherd your projects from idea through delivery into production
- Bring new ideas to the table some of our best innovations originate within the team
Technologies We Use
- Languages: SQL Python
- Infrastructure: AWS Docker Kubernetes Apache Airflow Apache Spark Apache Kafka
- Databases: Snowflake Trino/Starburst Redshift MongoDB Postgres MySQL
- Others: Tableau (as a business intelligence solution)
Qualifications :
- Bachelors/Masters degree in Computer Science or relevant technical degree
- 10 years of professional software engineering experience
- Strong proficiency with data languages such as Python and SQL
- Strong proficiency working with data processing technologies such as Spark Flink and Airflow
- Strong proficiency working of RDMS/NoSQL/Big Data solutions (Postgres MongoDB Snowflake etc.)
- Solid understanding of streaming solutions such as Kafka Pulsar Kinesis/Firehose etc.
- Solid understanding of Docker and Kubernetes
- Solid understanding of ETL/ELT and OLTP/OLAP concepts
- Solid understanding of columnar/roworiented data structures (e.g. Parquet ORC Avro etc.)
- Solid understanding of Apache Iceberg or other open table formats
- Proven ability to transform raw unstructured/semistructured data into structured data in accordance to business requirements
- Solid understanding of AWS Linux and infrastructure concepts
- Proven ability to diagnose and address data abnormalities in systems
- Proven ability to learn quickly make pragmatic decisions and adapt to changing business needs
- Experience building data warehouses using conformed dimensional models
- Experience building data lakes and/or leveraging data lake solutions (e.g. Trino Dremio Druid etc.)
- Experience working with business intelligence solutions (e.g. Tableau etc.)
- Experience working with ML/Agentic AI pipelines (e.g. Langchain LlamaIndex etc.)
- Understands Domain Driven Design concepts and accompanying Microservice Architecture
- Passion for data analytics or machine learning.
- Focus on value: shipping software that matters to the company and the customer
Bonus Points
- Experience working with vector databases
- Experience working within a retail or ecommerce environment.
- Proficiency in other programming languages such as Scala Java Golang etc.
- Experience working with Apache Arrow and/or other inmemory columnar data technologies
Supervisory Responsibility
- Provide mentorship to team members on adopted patterns and best practices.
- Organize and lead agile ceremonies such as daily standups planning etc
Additional Information :
EEO STATEMENT
Wiser Solutions Inc. is an Equal Opportunity Employer and prohibits Discrimination Harassment and Retaliation of any kind. Wiser Solutions Inc. is committed to the principle of equal employment opportunity for all employees and applicants providing a work environment free of discrimination harassment and retaliation. All employment decisions at Wiser Solutions Inc. are based on business needs job requirements and individual qualifications without regard to race color religion sex national origin family or parental status disability genetics age sexual orientation veteran status or any other status protected by the state federal or local law. Wiser Solutions Inc. will not tolerate discrimination harassment or retaliation based on any of these characteristics.
#LIRemote
Remote Work :
Yes
Employment Type :
Fulltime
Employment Type
Remote
