Data Engineer

Responsibilities
Design and automate essential data pipelines and inputs ensuring seamless integration
with downstream analytics and production systems.
Collaborate with crossfunctional teams to integrate new functionalities into existing
data pipelines including lower test environments to help validate and assess impact
prior to Production integration where applicable.
Implement data governance and quality processes to ensure the integrity and accuracy
of data throughout its lifecycle.
Monitor data systems and processes to identify issues and proactively implement
improvements to prevent future problems.
Participate in code reviews with senior developers prior to pushing the code into
production to ensure meeting accuracy and best practice standards.
Implement data pipeline Directed Acyclic Graphs (DAGs) and maintenance DAGs.
Configure and setup DAGs based on the data to run Spark commands in parallel and
sequential.
Perform unit testing using test cases and fix any bugs.
Optimize code to meet product SLAs
Support multiple projects and communicate with stakeholders in various organizations.
This includes regularly providing status updates developing timelines providing
insights etc.
Key Skills
Bachelors Degree in Computer Science Data Science Analytics or related field
3 years of experience with the following:
Coding in Python PySpark and SQL
Hive data storage technologies
Working within cloudbased infrastructures and tools such as AWS EC2 GitLab and
Airflow.

Working within the Software Development Life Cycle framework and applying
software development best practices
Building monitoring checks and tools to ensure infrastructure and related processes are
working as expected
Solid understanding of system design data structures and performance optimization
techniques
Excellent problem solving skills and attention to detail
Wellorganized and able to handle and prioritize multiple assignments
Able to communicate effectively both orally and in writing
(Preferred) 2 years experience with visualization and reporting tools e.g. Tableau
(Preferred) Experience deploying and maintaining Machine Learning models within
Production environments
(Preferred) Experience working with Jira Confluence and Smartsheets
(Preferred) Experience with Alteryx Databricks platform