Data Engineer 2
Data Engineer 2
Posted on :
09-03-2024
Employer Active
1 Vacancy
The job posting is outdated and position may be filled
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Role Overview:
As a Data Engineer 2 you will be responsible for designing implementing and maintaining scalable data pipelines and systems. You will collaborate closely with crossfunctional teams to understand data requirements develop robust solutions and ensure the reliability and performance of our data infrastructure. This role offers an exciting opportunity to work with cuttingedge technologies in a dynamic and fastpaced environment.
Key Responsibilities:
- Design develop and optimize data pipelines to collect process and store large volumes of structured and unstructured data.
- Implement ETL processes to transform raw data into actionable insights ensuring data quality and integrity throughout the process.
- Manage and maintain data warehouse systems including AWS Redshift to support analytical and reporting needs.
- Utilize Apache Airflow for workflow orchestration and scheduling of data pipelines.
- Work with streaming data technologies such as Apache Kafka to handle realtime data ingestion and processing.
- Collaborate with data scientists and analysts to understand their requirements and implement solutions for advanced analytics and machine learning.
- Monitor system performance troubleshoot issues and implement optimizations to ensure scalability reliability and efficiency.
- Stay updated on industry trends and best practices in data engineering and recommend new technologies and methodologies to enhance our data infrastructure.
Required Skills and Qualifications:
- Bachelors degree in Computer Science Engineering or related field.
- 36 years of experience in data engineering roles with a focus on building and maintaining data pipelines and systems.
- Proficiency in programming languages such as Python and Scala for data processing and automation.
- Strong experience with cloud platforms particularly AWS including services like Redshift EMR Athena and S3.
- Handson experience with workflow orchestration tools such as Apache Airflow.
- Familiarity with streaming data technologies like Apache Kafka for realtime data processing.
- Solid understanding of database concepts SQL and data modeling techniques.
- Experience working in Agile development environments and using version control systems like Git.
- Excellent problemsolving skills and the ability to work effectively in a collaborative team environment.
- Strong communication and interpersonal skills with the ability to effectively convey technical concepts to nontechnical stakeholders.
data engineering,aws,amazon redshift,etl,spark,kafka,athena,data modeling,data warehouse,data analysis,airflow,python,sql,glue,aws lambda,cdc,apache kafka
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
