Data Engineer

• Background in Computer Science Computer Engineer or Data Science;
• At least 35 years of total IT experience and 2 years of experience in data engineering;
• Working knowledge of programming languages such as SQL and Python;
• Experience in building data pipelines with Python and or Apache Spark
• Experience in working with Infrastructure as Code like Cloudformation or Terraform
• Experience in cloud data warehouse such as AWS Redshift Teradata Oracle Exacdata Machine or Cloudera Enterprise Data Hub;
• Experience in data model designing such as Star Schema or Snowflake;
• Familiar with big data streaming tools and database frameworks such as Apache NiFi Storm Spark Hadoop Tableau;
• Good writing communication and presentation skills;
• Fast learner and ability to work independently.

• Exposure in Hadoop or big data.
• Agile Scrum / Devops knowledge

• Design develop document and implement endtoend data pipelines and data integration processes both batch and realtime;
• Perform data analysis data profiling data cleansing data lineage data mapping and data transformation;
• Develop ETL / ELT jobs and workflows and deployment of data solutions;
• Monitor recommend develop and implement ways to improve data quality including reliability efficiency and cleanliness to optimize and finetune processes;
• Recommend execute and deliver best practices in data management and data lifecycle processes including modular development of data processes coding and configuration standards error handling and notification standards auditing standards and data archival standards;
• Collaborate with different stakeholders to understand data needs gather requirements and implement data solutions to deliver business goals;
• Provide technical support for any data issues and change requests document all investigations findings recommendations and resolutions.

Industry

• Information Technology & Services

Employment Type

cloudera enterprise data hub,spark,apache nifi,aws redshift,big data,hadoop,tableau,teradata,snowflake,storm,cloudformation,aws,sql,oracle exacdata machine,python,devops,agile scrum,terraform,star schema