Data Engineer with PySpark

Bangalore - India

Monthly Salary: Not Disclosed

Posted on: 2 days ago

Vacancies: 1 Vacancy

Job Summary

Key Responsibilities:

Design develop and maintain scalable data pipelines using PySpark
Build and manage data transformation workflows using DBT (Data Build Tool)
Develop test and deploy SQL-based data models in DBT
Process large volumes of structured and unstructured data
Implement ETL/ELT pipelines and optimize performance
Ensure data quality through testing validation and monitoring (DBT tests)
Collaborate with data analysts data scientists and business stakeholders
Integrate data from multiple sources (APIs databases cloud platforms)
Optimize queries and data models for performance and scalability
Maintain documentation for data models pipelines and workflows

Required Skills:

Strong experience in Python and PySpark
Hands-on experience with DBT (Data Build Tool)
Proficiency in SQL and data transformation techniques
Good understanding of Apache Spark architecture
Experience with modern data warehouses (Snowflake Big Query Redshift)
Knowledge of ETL/ELT pipeline development
Familiarity with version control tools (Git)
Experience with orchestration tools like Airflow

Key Responsibilities: Design develop and maintain scalable data pipelines using PySpark Build and manage data transformation workflows using DBT (Data Build Tool) Develop test and deploy SQL-based data models in DBT Process large volumes of structured and unstructured data Implement ETL/ELT pip...