Big data with spark and scala

Irving, TX - USA

Monthly Salary: Not Disclosed

Posted on: Yesterday

Vacancies: 1 Vacancy

Job Summary

Job Description:

Seeking a Senior Python Data Engineer with 3 to 5 years of experience skilled in Apache Spark and Python to design and implement scalable data processing solutions
Develop and maintain robust data pipelines using Apache Spark and Python
Collaborate with data scientists and analysts to optimize data workflows
Ensure data quality and integrity across large datasets Implement data transformations and aggregations for analytics and reporting
Work with cross-functional teams to integrate data solutions into existing systems
Monitor and troubleshoot data processing jobs to ensure reliability and performance
Stay updated with emerging technologies and best practices in big data and Python development

Roles and Responsibilities

Design develop and optimize scalable data processing applications leveraging Apache Spark
Write clean efficient and reusable Python code to support data engineering tasks
Analyze complex data sets to identify trends and insights for business use
Collaborate with stakeholders to gather requirements and deliver data solutions that meet business needs
Mentor junior team members and contribute to sharing knowledge within the team
Participate in code reviews and ensure adherence to coding standards and best practices
Manage data pipeline deployments and monitor their health in production environments

Mandatory Certificate

DASCA Big Data Associate Big Data Analyst
DASCA Big Data Associate Big Data Engineer
DASCA Big Data Senior Big Data Analyst
DASCA Big Data Senior Big Data Engineer
IIBA Business Data Analytics Certification

Job Description: Seeking a Senior Python Data Engineer with 3 to 5 years of experience skilled in Apache Spark and Python to design and implement scalable data processing solutions Develop and maintain robust data pipelines using Apache Spark and Python Collaborate with data scientists and analysts...