Data Engineer
Data Engineer
Posted on :
15-02-2024
Employer Active
1 Vacancy
The job posting is outdated and position may be filled
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Req ID : 2536385
We are seeking an experienced Data Engineer with expertise in both AWS and Azure cloud platforms. As a key member of our team you will be responsible for designing building and maintaining scalable data solutions to support our organisations datadriven initiatives. You will work closely with crossfunctional teams to understand business requirements architect data solutions and implement best practices for data management ETL (Extract Transform Load) data warehousing and analytics.
Contract: 6 Months (Fulltime Onsite)
Key Responsibilities:
Cloud Platform Expertise: Utilise your deep understanding of both AWS and Azure cloud platforms to architect implement and optimise data solutions that leverage the strengths of each platform.
Data Architecture Design: Design and implement robust data architectures that span multiple cloud environments ensuring scalability reliability and performance.
ETL Development: Develop test and optimise ETL processes to extract data from various sources transform it according to business requirements and load it into target systems such as data warehouses or data lakes.
Data Integration: Integrate data from disparate sources and systems across AWS and Azure environments to provide a unified view of data for analytics reporting and other business needs.
Big Data Technologies: Leverage big data technologies and frameworks such as Apache Spark Hadoop Kafka etc. to process and analyze large volumes of data efficiently.
Data Governance and Security: Implement data governance policies security measures and compliance controls to protect sensitive data and ensure regulatory compliance across AWS and Azure environments.
Performance Optimization: Monitor and optimize data pipelines queries and processes to improve performance reduce latency and maximize throughput.
Data Quality Assurance: Implement data quality checks validation rules and monitoring processes to ensure the accuracy completeness and consistency of data across different systems and stages of the data lifecycle.
Collaboration and Communication: Collaborate with business stakeholders data scientists analysts and other IT teams to understand requirements provide technical guidance and deliver solutions that meet business objectives.
Documentation and Best Practices: Document data architectures design decisions and best practices for data engineering on AWS and Azure and provide training and support to other team members to promote knowledge sharing and improve data engineering capabilities across the organization.
Qualifications:
- Bachelors or Masters degree in Computer Science Information Systems or related field.
- 5 years of experience in data engineering with expertise in both AWS and Azure cloud platforms.
- Strong proficiency in AWS services such as S3 EC2 Redshift Glue Athena EMR etc. and Azure services such as Azure Data Factory Azure SQL Database Azure Synapse Analytics Azure Databricks etc.
- Handson experience with ETL tools data modelling SQL Python or other programming languages used in data manipulation and analysis.
- Experience with big data technologies such as Apache Spark Hadoop Kafka etc. and familiarity with data lakes NoSQL databases and realtime data processing.
- Solid understanding of data governance security and compliance requirements with experience implementing best practices in a multicloud environment.
- Excellent problemsolving skills attention to detail and ability to work independently and collaboratively in a dynamic fastpaced environment.
- Strong communication and interpersonal skills with the ability to effectively communicate technical concepts to both technical and nontechnical stakeholders.
- Relevant certifications such as AWS Certified Solutions Architect AWS Certified Big Data Specialty Microsoft Certified: Azure Data Engineer Associate etc. are highly desirable.
apache spark,data modeling,sql,python,s3,ec2,amazon redshift,glue,athena,electronic medical record (emr),azure,aws,nosql,hadoop,kafka,data integration,etl,data architecture,big data technologies,data quality assurance,data governance,web performance optimization,big data,cloud,data engineering,analytics
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
