Senior Big Data Engineer

Warsaw - Poland

Monthly Salary: Not Disclosed

Posted on: 8 hours ago

Vacancies: 1 Vacancy

Department:

Engineering

Job Summary

Design build and maintain scalable data pipelines using Python/PySpark and AWS services.
Develop test and deploy dbt models ensuring quality through unit tests and clear documentation.
Build and integrate data connectors (FTP API JDBC etc.).
Implement data ingestion transformation and orchestration workflows using AWS Glue AppFlow Lake Formation Transfer Family MWAA or Argo.
Use AWS CDK for infrastructure-as-code and deployment automation.
Apply best practices for CI/CD version control (Git) testing and data quality monitoring.
Collaborate with cross-functional teams to understand data requirements and optimize data flows.
Troubleshoot performance issues and ensure secure reliable and high-performing data pipelines.

Qualifications :

Strong hands-on experience with dbt (core concepts models tests documentation).
Proficiency in Python and/or PySpark for ETL/ELT development.
Expertise in building connectors via FTP API JDBC or other integration patterns.
Solid experience with AWS data services: Glue AppFlow Lake Formation Transfer Family S3 MWAA etc.
Experience with workflow orchestration tools such as MWAA or Argo. Knowledge of AWS CDK for infrastructure automation.
Experience with Git CI/CD tools and automated testing practices.
Strong understanding of data quality monitoring validation tests and logging.
Good communication skills and ability to work in cross-functional agile teams.

Remote Work :

Employment Type :

Full-time

Design build and maintain scalable data pipelines using Python/PySpark and AWS services.Develop test and deploy dbt models ensuring quality through unit tests and clear documentation.Build and integrate data connectors (FTP API JDBC etc.).Implement data ingestion transformation and orchestration wor...

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

Nagarro

Nagarro helps future-proof your business through a forward-thinking, fluidic, and CARING mindset. We excel at digital engineering and help our clients become human-centric, digital-first organizations, augmenting their ability to be responsive, efficient, intimate, creative, and susta ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click