Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Duties & Responsibilities
Data Engineer
A leading financial services and insurance organization is seeking an experienced Data Engineer with a strong focus on DevOps to join its Data Engineering Department. This role is instrumental in designing building and maintaining robust scalable data pipelines and systems that support data-driven decision-making across the enterprise.
You will work closely with cross-functional teams to enable seamless integration with AWS services drive containerization through Docker and Kubernetes and manage performance of Apache Spark and Kafka deployments. If youre passionate about optimizing data operations and leveraging modern DevOps and cloud practices wed like to meet you.
Key Responsibilities:
- Design develop and deploy scalable and efficient data pipelines tailored to business needs.
- Collaborate with the DevOps team to build and manage CI/CD pipelines using AWS tools such as CodePipeline CodeBuild and CodeDeploy.
- Containerize and orchestrate data processing applications using Docker and Kubernetes.
- Manage and optimize deployments of Apache Spark and Kafka to support high-performance data processing.
- Monitor data pipelines for performance reliability and error reduction.
- Apply security and compliance best practices in all data engineering workflows.
- Evaluate and introduce new tools and technologies to improve data engineering productivity and system performance.
- Provide technical support to team members resolving issues related to data workflows and infrastructure.
Required Skills & Qualifications:
- Bachelors degree in Computer Science Software Engineering or a related field.
- Minimum 5 years of experience in data engineering with hands-on experience in pipeline development and DevOps integration.
- Strong knowledge and hands-on experience with:
- Apache Spark
- Databricks
- Oracle
- Kafka deployment and performance tuning
- Proficiency in AWS DevOps tools including CodePipeline CodeBuild CodeDeploy and CodeStar.
- Experience using containerization and orchestration tools (Docker and Kubernetes).
- Familiarity with other data processing frameworks such as Hadoop Apache NiFi or Apache Beam.
- Excellent troubleshooting and problem-solving skills.
- Strong communication skills with the ability to bridge the gap between technical teams and business stakeholders.
Location:
- Johannesburg Gauteng
Workplace Type:
- Hybrid
Job Type:
- Contract
Experience Type:
- Senior
We encourage you to apply - Kivara Rajgopal on or via
Employment Type
Full-Time
