ETL Databricks with AWS (PySpark/SQL)
ETL Databricks with AWS (PySpark/SQL)
Posted on :
08-07-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Develop and maintain a metadata driven generic ETL framework for automating ETL code Design build and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS .Ingest data from a variety of structured and unstructured sources (APIs RDBMS flat files streaming).Develop and maintain robust data pipelines for batch and streaming data using Delta Lake and Spark Structured data quality checks validations and logging pipeline performance cost and with data analysts BI and business teams to deliver fit for purpose data modeling efforts (star snowflake schemas) de norm tables approach and assist with data warehousing with orchestration tools Databricks Workflows to schedule and monitor best practices for version control CI/CD and collaborative developmentSkillsHands-on experience in ETL/Data Engineering expertise in Databricks (PySpark SQL Delta Lake) Databricks Data Engineer Certification preferredExperience with Spark optimization partitioning caching and handling large-scale in SQL and scripting in Python or understanding of data lakehouse/medallion architectures and modern data working with cloud storage systems like AWS S3Familiarity with DevOps practices Git CI/CD Terraform debugging troubleshooting and performance-tuning skills.
Employment Type
Full-Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
