Data Engineer Architect

About the Role:

Key Responsibilities:

• Design and implement scalable and faulttolerant data streaming and ingestion pipelines using Apache Kafka.
  
• Architect and develop a cloudbased data warehousing solution using technologies using such as Apache Hadoop HDFS Snowflake. Apache Hive or similar technologies for the storage and analysis of structured and unstructured data.
  
• Lead the development of data transformation and modeling processes using DBT Apache Spark Apache Beam or similar technologies.
  
• Lead design and implementation of Reporting / Visualization efforts using Apache Superset Apache Zeppelin Metabase Looker or similar technologies.
  
• Collaborate with frontend developers to embed interactive dashboards and reports using Apache Superset or similar tools into a React frontend.
  
• Ensure data security governance and compliance requirements are met by implementing access controls and auditing mechanisms using Apache Ranger or Apache Sentry or similar technologies.
  
• Mentor and guide junior team members provide technical leadership and contribute to the overall technical strategy and roadmap for the analytics platform.
  

KINDLY TAKE NOTE THAT THE RECRUITMENT AND SELECTION PROCESS WILL INVOLVE

PSYCHOMETRIC ASSESSMENTS.

Requirements

Required Skills and Experience:

• Experience in building realtime analytics platforms.
  
• 9 Experience in data engineering and data architecture with a focus on a cloudbased data warehousing solutions around structured (dimensional models) and unstructured data.
  
• Proficiency in Apache Kafka for data streaming and ingestion.
  
• Experience with cloudbased data warehousing solutions such as Apache Hadoop HDFS Snowflake Amazon Redshift Amazon S3 or Google Cloud Storage.
  
• Proficiency in the programming languages as relevant for the platforms involved such as Java SQL Python or Scala.
  
• Handson experience with Apache Spark or Apache Beam for largescale data processing and transformation.
  
• Experience with data visualization and exploration tools such as Apache Superset Apache Zeppelin Metabase Looker or similar platforms.
  
• Familiarity with security and governance tools and practices in the Hadoop ecosystem such as Apache Ranger or Apache Sentry.
  
• Excellent communication skills and the ability to work effectively in a collaborative team environment.
  
• Bachelors or Masters degree in Computer Science Engineering or a related field if preferred but not mandatory.