Data Engineer with Pyspark
Job Location:
Monthly Salary:
Experience Required:
Posted:
Application Deadline:
Vacancies:
Job Summary
- Design develop and maintain scalable and reliable data processing solutions using PySpark.
- Build and manage robust batch and streaming data pipelines.
- Develop efficient data transformation and processing solutions using Python PySpark and SQL.
- Design develop and optimize data models to support scalable and high-performance data processing.
- Work with large datasets and complex data processing workloads.
- Monitor troubleshoot and optimize data pipelines for performance reliability and scalability.
- Implement data quality validation and error-handling mechanisms across data pipelines.
- Collaborate with engineering architecture and business teams to deliver reliable data solutions.
- Contribute to Agile development practices and continuously improve data engineering standards and processes.
- 5 years of professional experience in Data Engineering or a related field.
- Strong hands-on experience with PySpark as a core data processing technology.
- Strong proficiency in Python PySpark and SQL.
- Hands-on experience developing and maintaining batch and streaming data pipelines.
- Experience working with large-scale data processing and transformation.
- Good understanding of data modeling data integration and data optimization.
- Experience with cloud-based data platforms and modern data engineering architectures.
- Experience with data quality validation troubleshooting and performance optimization.
- Familiarity with CI/CD and modern software engineering practices.
- Experience working in Agile development and delivery environments.
- Strong communication collaboration analytical and problem-solving skills.
- Develop scalable and high-performance data pipelines using PySpark.
- Build efficient data transformations using Python PySpark and SQL.
- Process and manage large volumes of structured and unstructured data.
- Optimize Spark jobs data processing performance and resource utilization.
- Troubleshoot complex data pipeline and processing issues.
- Design reliable and maintainable data engineering solutions.
- Implement data quality and validation processes.
- Work effectively with technical and business stakeholders.
- Apply best practices for code quality scalability maintainability and reliability.
- Continuously improve data engineering processes and solutions.
- Opportunity to work on enterprise-scale data engineering initiatives in Spain.
- Exposure to modern data platforms PySpark cloud technologies and large-scale data processing.
- A collaborative Agile environment focused on technical excellence and innovation.
- Opportunities to work on complex data pipelines and data transformation solutions.
- Continuous learning and opportunities for technical and professional growth.
- A culture focused on quality ownership scalability and sustainable data engineering solutions.
As a Data Engineer with PySpark you will: Design develop and maintain scalable and reliable data processing solutions using PySpark. Build and manage robust batch and streaming data pipelines. Develop efficient data transformation and processing solutions using Python PySpark and SQL. Design develop and optimize data models to support scalable and high-performance data processing. Work with large datasets and complex data processing workloads. Monitor troubleshoot and optimize data pipelines for performance reliability and scalability. Implement data quality validation and error-handling mechanisms across data pipelines. Collaborate with engineering architecture and business teams to deliver reliable data solutions. Contribute to Agile development practices and continuously improve data engineering standards and processes. What You Bring to the Table: 5 years of professional experience in Data Engineering or a related field. Strong hands-on experience with PySpark as a core data processing technology. Strong proficiency in Python PySpark and SQL. Hands-on experience developing and maintaining batch and streaming data pipelines. Experience working with large-scale data processing and transformation. Good understanding of data modeling data integration and data optimization. Experience with cloud-based data platforms and modern data engineering architectures. Experience with data quality validation troubleshooting and performance optimization. Familiarity with CI/CD and modern software engineering practices. Experience working in Agile development and delivery environments. Strong communication collaboration analytical and problem-solving skills. You Should Possess the Ability to: Develop scalable and high-performance data pipelines using PySpark. Build efficient data transformations using Python PySpark and SQL. Process and manage large volumes of structured and unstructured data. Optimize Spark jobs data processing performance and resource utilization. Troubleshoot complex data pipeline and processing issues. Design reliable and maintainable data engineering solutions. Implement data quality and validation processes. Work effectively with technical and business stakeholders. Apply best practices for code quality scalability maintainability and reliability. Continuously improve data engineering processes and solutions. What We Bring to the Table: Opportunity to work on enterprise-scale data engineering initiatives in Spain. Exposure to modern data platforms PySpark cloud technologies and large-scale data processing. A collaborative Agile environment focused on technical excellence and innovation. Opportunities to work on complex data pipelines and data transformation solutions. Continuous learning and opportunities for technical and professional growth. A culture focused on quality ownership scalability and sustainable data engineering solutions. Lets Connect: Want to discuss this opportunity in more detail Feel free to reach out. Recruiter: Hema Murali Phone:; Extn :148 Email: LinkedIn:
