Senior AWS Data Engi....
Senior AWS Data Engineer
Posted on :
01-12-2023
Employer Active
1 Vacancy
The job posting is outdated and position may be filled
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Req ID : 2377504
This is a remote position.
We are seeking a skilled AWS Data Engineer with a focus on ETL data pipeline design and Redshift expertise. The ideal candidate will have a strong background in designing and implementing data pipelines using AWS Glue orchestrating workflows with Airflow and handling various data ingestion patterns such as batch streaming (Kafka/Kinesis) and API integration. The candidate should also be proficient in scripting languages particularly Python using tools like AWS Glue (Spark) and have experience in building and enhancing API ingestion pipelines using AWS API Gateway.
As an AWS Data Engineer you will collaborate with Solution Architects to enhance the data ingestion framework making it metadatadriven and supporting multiple ingestion patterns including batch processing streaming data and API integration. You should possess the ability to understand and explain data ingestion processes from diverse sources like files databases and applications.
Requirements
Design and implement data pipelines using AWS Glue and orchestrate workflows with Airflow for batch streaming and API ingestion.
Develop API ingestion pipelines using AWS API Gateway.
Collaborate with Solution Architects to enhance the data ingestion framework using AWS native services.
Build and enhance Python PySparkbased frameworks for ingestion.
Utilize AWS Data Services such as Glue EMR Airflow CloudWatch Lambda Step Functions and Event Triggers.
Act as a senior engineer with a sole interaction point with different business functional teams.
Requirements:
Experience/Skills:
Minimum of 8 years of IT experience.
Proficient in AWS Glue Lambda Airflow/Step Functions PySpark Python SQL and DBT.
Extensive experience in designing and implementing data pipelines ingestion and Redshift.
Strong scripting skills preferably in Python using Cargill standard tools such as AWS Glue (Spark).
Expertise in designing and implementing API ingestion pipelines using AWS API Gateway.
Familiarity with metadatadriven frameworks and supporting multiple ingestion patterns.
Strong understanding of data ingestion from various sources like files databases and applications.
Handson experience in building and enhancing Python PySparkbased frameworks for ingestion.
Expertise in AWS Data Services and related technologies.
Experience working as a React.js Developer. In-depth knowledge of JavaScript, CSS, HTML, and front-end languages. Knowledge of REACT tools including React.js, Webpack, Enzyme, Redux, and Flux. Experience with user interface design. Experience with browser-based debugging and performance testing software. Excellent troubleshooting skills. Good project management skills.
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
