Senior Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
RESPONSIBILITIES:
Data Pipeline Development:
- Design develop and maintain robust and scalable data pipelines.
- Implement ETL (Extract Transform Load) processes to integrate data from various sources.
- Ensure data pipelines are optimized for performance and reliability.
- Work closely with Machine Learning System analysts and other stakeholders to understand data needs.
- Perform analyses on large sets of data to extract actionable insights that will drive decisions with a particular focus on our marketing and sales strategy. Engineer novel data sets and features that shed new light on the dynamics of consumer choices.
- Provide technical support for data-related issues and challenges.
- Collaborate with the Lead Data Engineer to implement data solutions and improvements.
Data Integration and Storage:
- Integrate data from different sources including databases APIs and external data providers.
- Ensure data is clean consistent and available for analysis.
- Collaborate with other teams to understand data requirements and deliver integrated solutions.
- Manage and maintain data storage solutions including databases data lakes and data warehouses.
- Ensure data storage systems are efficient secure and scalable.
- Implement best practices data governance and security measures to protect sensitive information as well asdata storage and management.
Data Quality Assurance:
- Implement processes to ensure data quality accuracy and consistency.
- Monitor data pipelines for errors and address issues promptly.
- Conduct regular data validation and verification.
- Optimize data workflows for performance reliability and scalability.
- Perform data profiling cleansing and transformation to ensure data quality and integrity.
- Optimize data processing and storage systems for efficiency and performance.
- Continuously evaluate and improve data engineering processes and tools.
- Stay updated with the latest trends and technologies in data engineering.
- Bachelors degree in Computer Science Information Technology Data Science or a related field.
- At least 5 years experience in data engineering data architecture or a related role.
- Experience with ETL processes data integration and data pipeline development.
- Experience in a financial institution or similar environment is preferred.
- Strong programming skills in languages such as Python Java or Scala.
- Proficiency in SQL and database management.
- Experience with big data technologies (e.g. Hadoop Spark) and cloud data platforms (e.g. AWS Azure GCP).
- Knowledge of data warehousing solutions (e.g. Redshift BigQuery Snowflake).
- Familiarity with data visualization tools (e.g. Tableau Power BI) is an added advantage.
COMPETENCIES REQUIREMENTS:
Technical:
- Cloud Computing
- Cybersecurity Management
- Data Analysis
- Database Management
- Digital Transformation
- Information Security Management
- IT Application Support
- IT Governance and Compliance
- IT Infrastructure Management
- IT Service Management (ITSM)
- Mobile Device Management
- Network Administration
- Programming and Scripting
- Project Management
- Quality Assurance
- Regulatory Compliance
- Software Development
Behavioural:
- Entrepreneurial Mindset
- Excellence
- Execution
- Energy
- Empathy
- Evolution
- Emotional Intelligence
- Business Acumen
- Decision-Making
- Result Oriented
- Communication written & verbal
- Stakeholder Management
- Analytical Thinking
- Managing Risk
- Service Orientation
What to Expect in the Hiring Process:
- A preliminary phone call with the recruiter
- Technical interview
- Assessment
- Interview with Senior members of the team
- Cultural and Behavioural Fit Interview with a member of the Executive team.
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
