Data Engineer

Data Engineer

Oldwick NJ

MUST HAVE AZURE & AIRFLOW.

One or two days a week in the office.

Responsibilities:

• Build data pipelines / data orchestrator and data structures for existing operational data stores Azure data lake & data warehouse.
• Develop proofofconcepts to evaluate new technologies and suggest improved and better ways to rearchitect existing data loading processes and data structures.
• Design and develop ELT / ETL to load data from internal sources into operational data stores Azure data lake and data warehouse.
• Create APIs data structures and schemas to store and retrieve data.
• Create and maintain data mappings data definitions architecture and data flow diagrams.
• Implement solutions to validate and test data quality.
• Build proofofconcepts to determine viability of new processes and technologies.
• Deploy and manage code in nonprod and prod environments.
• Troubleshoot data related issues and fix defects.
• Optimize and performance tune databases and queries.
• Participate in code reviews and train other team members.
• Document code and create code templates based on industry best practices.

Requirements:

• Bachelors Degree and/or equivalent experience required.
• Minimum 5 years of experience working as a Data Engineer ETL Engineer Data/ETL Architect or similar roles.
• 5 years of solid continuous experience in Python.
• 3 years of continuous experience with Airflow.
• 3 years of experience with Azure Data Factory (ADF) or similar.
• 3 years of experience working with relational databases: Oracle SQL Server PostgreSQL or similar.
• 3 years of experience working with NoSQL databases: MongoDB Cosmos DB DocumentDB or similar.
• 3 years of experience writing SQL code.
• 3 years of experience in Kimball Dimensional Modeling (StarSchema).
• 2 years of experience in columnar databases: Snowflake Azure Synapse or similar.
• 2 years of Azure experience.