Data Engineer
Data Engineer
Posted on :
08-02-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Req ID : 2516418
JD:
Role: Sr / Lead Data Engineer
Experience: 5 to 8 years
Job Location : Hyderabad Work from Office
Mode of employment: Permanent
Notice period: Immediate to 15 days ONLY.
Skills and Responsibilities.
Strong programming skills in Python Scala or SQL as well as Pyspark.
Architect and implement data pipelines using Apache Spark for efficient data processing and transformation.
Develop and maintain data workflows using Apache Airflow to automate data pipeline orchestration and scheduling.
Create and optimize SQL queries for data extraction transformation and loading (ETL) processes.
Harness the power of Python for data manipulation scripting and automation.
Manage and optimize data storage and processing on AWS EMR Redshift and Azure Synapse.
Collaborate with teams to design and maintain Azure Data Factory (ADF) pipelines for data integration.
Leverage DBT for data modelling and version control ensuring data accuracy and consistency.
Integrate data from diverse sources using Fivetran ensuring seamless data ingestion.
Continuously improve data quality performance and reliability across the data ecosystem.
Monitor and troubleshoot data pipelines ensuring data availability and integrity.
Implement data security best practices and ensure compliance with data privacy regulations.
data,pipelines,apache,azure,processing,python,sql,skills,transformation,aws
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
