Data Operations Engineer
Data Operations Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Key roles and responsibilities:
- Undertake the processing of structured and unstructured data to utilize in data science activities.
- Support the data analysis of large information to discover trends and patterns and provide feedback to the business.
- Assist in designing and implementing scalable and robust processes for ingesting and transforming datasets.
- Design implement and support the creation and maintenance of data pipelines from a multitude of sources.
- Ingest large complex data sets that meet functional and nonfunctional requirements.
- Request For Change (RFC) management in Amazon Managed Services (AMS) (logging escalation resolution)
- Provisioning infrastructure in AWS (via RFC process CloudFormation Console)
- AWS Security Management (authoring IAM security policies logging security requests user management/permissions)
- Infrastructure As Code (development of CloudFormation templates)
- Development deployment and management of CICD (Continuous Integration Continuous Deployment ) pipelines.
- Linux and Windows System Administration within AWS (SSH RDP).
- Design and build bulk and delta data lift patterns for optimal extraction transformation and loading of data.
- Supports the organisations cloud strategy and aligns to the data architecture and governance including the implementation of these data governance practices.
- Work with various stakeholders across the organization to understand data requirements and apply technical knowledge of data management to solve key business problems.
- Provide support in the operational environment with all relevant support teams for data services.
Knowledge skills and attributes
- Working with large volumes of structured and unstructured data and leveraging them to build artificial intelligence (AI)/machine learning (ML) and predictive modelling (PM) solutions through endtoend automated data pipelines.
- Deep and Extensive AWS knowledge and skills: EMR Lambda EC2 VPC Transit Gateway S3IAM Infrastructure As Code (e.g. CloudFormation) Code Commit Code Pipeline and Code Build.
- Advantageous AWS experience with DynamoDB RDS (MS SQL Oracle) Landing Zone Control Tower and AMS
- The candidate having DBA ability and knowledge across at least 2 platforms (for example TSQL SAS PSQL IBM VSAM DB2 etc.) will also be beneficial.
- Experience with the Python programming language.
- Experience with designing and implementing Cloud (AWS) solutions including the use of APIs available.
- Experience with Dev/OPS architecture implementation and operation would be advantageous.
- Some experience in applying SAFe/Scrum/Kanban methodologies would be advantageous.
- Knowledge of Engineering and Operational Excellence using standard methodologies. Best practices in software engineering data management data storage data computing and distributed systems to solve business problems with data.
- Strong AWS cost management billing resolution and cost optimization
Academic qualifications and certifications:
- Matric with a degree in Computer Science Business Informatics or Data Engineering
- 5 years IT work experience and 2 years of experience in a senior technical position in a relevant role e.g. System Administrator Network Engineer DevOps Engineer Cloud Engineer.
- 5 years of data engineering experience
- 3 years of experience with any data warehouse technical architectures ETL/ELT and reporting/analytics tools including but not limited to any of the following combinations (1) SSIS and SSRS (2) SAS ETL Framework (3) SAP ETL Framework (4) MongoDB ETL deployments (5) Apache Spark and Apache Hive deployments will be beneficial.
Employment Type
Full Time
