Data Engineer

• Mentor BI Developers, Data Engineers, and Analytics Engineers to aid in growth and ensure best practices are followed.
  
• Own production data ingestion and transformation processes, ensuring the successful, on-time execution of all pipelines.
  
• Develop, own, and ensure adherence to quality standards for ETL/ELT, Python, Talend, Snowflake, and Azure as well as documenting and training other teammates on these standards.
  
• Data engineering subject matter expert on cross-functional teams developing scalable solutions for new and modified data sources.
  
• Lead the design, development, and optimization of data pipelines using tools like Talend, Hadoop, Spark, and Azure Services and ingest/move data into SQL Servers, Data Lakes, or Snowflake.
  
• Design, modify, and implement data structures in Snowflake to support data ingestion, integration, and analytics.
  
• Curate and transform data into appropriate structures for analytics and data science purposes using SQL and Python scripting, and data transformation tools like Informatica and Talend.
  
• Design and implement processes to automate monitoring and alerting on source data quality, data ingestion and transformation processes, and the overall health of company or Client s data infrastructure.
  
• Monitor and ensure client s ETL pipeline is operational with ready to act remediation plans ready in case of risks involved
  

Requirements

• 5+ years of relevant data warehouse and data lake engineering experience
  
• 3+ years of Cloudera experience
  
• Expertise in data pipeline design and automation
  
• Advanced SQL and Azure expertise
  
• Expertise in API development
  
• Advanced expertise in one or more cloud data platforms like Oracle, Azure Data Lake, Azure SQL, or similar databases
  
• Experience with advanced ETL/ELT tool administration (code deployment, security, setup and configuration)
  
• Experience with enterprise ETL/ELT tools like Azure Data Factory, Data Bricks, Talend or other similar ETL/ELT tools
  
• Advanced expertise coding in Python including advanced data structures and libraries
  
• Drive to mentor and guide data engineers, BI developers, and data analysts
  
• Ambition to design and implement code from scratch and build new data infrastructure
  
• Enjoy supporting team members and helping solve problems that arise
  
• Enjoy a connected, collegial environment even though we are remote, hybrid, and on-site
  
• Expertise with documenting data definitions and code
  
• Experience deploying applications in Docker
  
• Experience with a cloud-based BI platform like Tableau, Grafana, or Power BI