Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Hybrid position
Minimum requirements:
- Relevant 3year tertiary qualification BSC In Computer Science / Information Systems
- 24 Years Senior Data Engineering experience
In depth knowledge and understanding of:
- Azure Databricks
- Data governance and data management frameworks
- BI and Warehouse design
- Movement of data into the Cloud using the following tools Apache Kafka Storm Flume for ingesting data or Amazon Web Services (AWS) Cloud Development Kit (CDK)
- Operating with realtime streams data warehouse queries JSON CSV raw data
- Scripting Data Pipelines for scheduled movement of data
- Designing and developing ETL/ELT processes and data pipelines Experience working with Azure DevOps
- Data visualization including PowerBI / Tableau
- Exploratory Data Analysis EDA
- SQL
- Microsoft SQL Server
- NoSQL
- Microsoft SQL Server
- Python (PySpark)
- C# Json calling APIs
- PowerShell
- Apache Spark
- Kafka
- Scala
- Splunk
- Elk Stack
- Data Modelling
- Data warehousing solutions
- Data pipelines
- Data Cleansing
- Terraform
Responsibilities:
- Work as part of an agile data engineering team
- Ensure that data systems meet the companys regulatory commitments and that data is reliable and efficient
- Support best business capabilities for high performance database solutioning
- Actively participate in team crossdiscipline and vendordriven collaboration sessions or forums to increase understanding of the working environment by contribution and participation in the relevant Data Governance Frameworks
- Partner with Technology Business and other stakeholders to ensure that database structures programmes and applications meet business delivery requirements
- Monitor adherence to processes which support the prescribed data architectural frameworks and ensure development/delivery teams align to the required standards and methodologies
- Design and implement scalable endtoend database solutions including:
- Addressing issues of data migration i.e. validation cleanup and mapping and consistently apply data dictionaries
- Data cleanse using databricks
- Component design and development
- Identification and resolution of production and application development constraints
- Integration of new technologies and software into the existing landscape
- Development data set processes for data modelling mining and production
- Optimal utilization of big data
- Documentation management and disaster recovery
Employment Type
Full Time
