Data Engineer

Job Summary

We are looking for an experienced Data Engineer to take ownership of our data infrastructures efficiency performance and reliability. The ideal candidate will be responsible for auditing our current systems identifying areas for improvement and implementing robust solutions to ensure our data is both performant and trustworthy. This role is critical in establishing a ground truth data source that will empower our data analytics and data science teams to derive actionable insights with confidence. You will be instrumental in building and maintaining a scalable and resilient data ecosystem that supports the companys strategic objectives.

Key Responsibilities

Infrastructure Auditing & Optimization

- Conduct comprehensive audits of the existing data infrastructure to assess its efficiency scalability and performance.

- Identify and analyze performance bottlenecks and propose and implement optimization strategies.

- Re-design and modernize data pipelines for greater scalability and reduced latency.

- Implement monitoring and alerting systems to proactively identify and address infrastructure issues.

Data Pipeline & ETL Development

- Design build and maintain robust and scalable ETL/ELT processes to ingest data from a wide variety of sources.

- Assemble large complex datasets that meet both functional and non-functional business requirements.

- Automate manual data processes to improve efficiency and reduce the potential for human error.

Ground Truth Data Source Management

- Establish and maintain a centralized reliable source of truth for all key business data.

- Implement rigorous data quality checks and validation processes to ensure data accuracy and consistency.

- Develop and enforce data governance best practices including data lineage and metadata management.

Collaboration & Support

- Work closely with data scientists data analysts and other stakeholders to understand their data requirements and provide them with the data they need.

- Build analytical tools and provide technical support to assist teams in leveraging the data infrastructure effectively.

- Act as a subject matter expert on data engineering best practices and advocate for their adoption across the organization.

Skills and Qualifications

The following skills and qualifications are required for this role:

Educational Background: A Bachelors degree in Computer Science Engineering or a related technical field.

Professional Experience: A minimum of 3-5 years of hands-on experience in a data engineering role.

Technical Proficiencies:

Programming: Advanced proficiency in Python with experience in either Java or Scala being a plus.

SQL: Expertise in writing complex highly-optimized SQL queries across large datasets.

Cloud Platforms: Demonstrable experience with at least one major cloud platform such as AWS (Redshift S3 EC2) Google Cloud Platform (BigQuery Dataproc) or Azure.

Big Data Technologies: Hands-on experience with big data tools like Apache Spark Hadoop and Kafka.

Data Warehousing: Experience with modern data warehousing solutions such as Snowflake Redshift or BigQuery.

Soft Skills:

Excellent problem-solving and analytical skills.

Strong communication and collaboration abilities with a knack for explaining complex technical concepts to non-technical audiences.

A proactive and self-motivated work ethic with a strong sense of ownership and a commitment to delivering high-quality results.

Preferred Qualifications

While not mandatory the following qualifications will be highly regarded:

A Masters degree in a relevant technical field.

Professional certifications in cloud technologies (e.g. AWS Certified Data Analytics Google Professional Data Engineer).

Experience with data orchestration tools such as Airflow or Prefect.

Familiarity with containerization technologies like Docker and Kubernetes.

A solid understanding of data modeling principles and best practices.

What We Offer

provides a competitive salary and benefits package a flexible work environment and a culture that fosters innovation and professional growth. We are an equal opportunity employer and value diversity at our company.

Required Experience:

IC