Data Engineer
Cape Town - South Africa
Job Summary
- Translate business requirements into technical data solutions.
- Participate in and contribute to architectural forums within the Business Intelligence (BI) team.
- Design and implement ETL/data pipeline architectures using tools like ADF (Azure data factory).
- Automate processes and design system architectures.
- Develop and manage Power BI semantic models including understanding DAX queries and cubes.
- Visualization and Dash boarding: translate metrics into visual Power BI reports
- Integrate data between legacy and modern systems.
- Create ad-hoc SQL scripts to support user queries.
- Deploy and manage data solutions on cloud platforms like Azure.
- Understanding of Data Warehouse modeling and Techniques: Data schemas Facts dimensions and building data warehouses.
- Writing complex SQL queries joins views and stored procedures to extract and manipulate data.
- Apply knowledge and practical experience in the Data Warehouse Life Cycle.
- Perform data modeling including dimensional multi-dimensional and relational models.
- Utilize TSQL for database management and querying.
- Support batch processing and scheduling tasks.
- Create and maintain technical documentation such as data architecture diagrams ETL workflows and system documentation to ensure maintainability.
- Participate in design peer and code reviews.
- Provide daily technical functional and operational support for existing BI solutions.
- Monitor data pipeline and infrastructure performance.
- Identify bottlenecks and optimize for scalability reliability and cost-efficiency.
- Troubleshoot and resolve data-related issues.
- Contribute to evolving the architecture towards modern platforms whether on-premise or cloud-based.
- Maintain working knowledge of Power BI reporting.
- Ensure technical skills stay relevant to emerging industry trends and organizational strategies including Azure Cloud Solutions and Artificial Intelligence initiatives.
Qualifications:
- Matric essential
- Degree in Computer Science / Engineering / Mathematics or related discipline (could be replaced with 10 years relevant experience)
- 4-5 years total experience (some experience may be replaced by post graduate qualifications if the practical component was relevant)
Desired Experience:
Data Engineering
- Data pipeline architecture and development
- ETL architecture and implementation
- Azure Data Factory (ADF)
- Process automation
- Integration between legacy and modern systems
- Batch processing and scheduling
- Data Warehouse lifecycle
SQL / Database Technologies
- Advanced/complex SQL
- T-SQL
- SQL joins
- Views
- Stored procedures
- Database querying and management
- Ad-hoc SQL scripting
Data Warehousing & Data Modeling
- Data Warehouse modelling techniques
- Data schemas
- Facts and dimensions
- Data Warehouse construction
- Dimensional modelling
- Multi-dimensional modelling
- Relational modelling
Power BI / Business Intelligence
- Power BI reporting
- Power BI semantic models
- DAX queries
- Cubes
- Power BI visualization and dashboard development
- Translating business metrics into visual reports
Cloud / Azure
- Deploying data solutions on Azure
- Managing cloud-based data solutions
- Azure Cloud Solutions
- Understanding the evolution from on-premise to modern/cloud-based architectures
Architecture
- Data/system architecture
- Designing scalable data solutions
- Architectural forums within a BI environment
- Modernising architecture
- Architecture diagrams and technical documentation
Required Skills:
Minimum 5 years of Project Management experience. At least 3 years experience managing Data Migration projects
Required Education:
5 to 7 years of professional software testing experience often with a mix of manual and automated qualifications and/or applicable training in application programming courses and/or International Software Testing with MS SQL Server and data access methods SQL and extensive work done on Database Technologies (MS SQL PostgreSQL MySQL) including familiarity with Stored working within Agile/Scrum and DevOps delivery cycles.