Senior PySpark Developer Data Engineering
Senior PySpark Developer Data Engineering
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Software Requirements:
- Proficiency in PySpark and Spark development.
- Experience with Unix and HDFS (Hadoop Distributed File System).
- Familiarity with libraries such as PyArrow.
- Working knowledge of SQL and databases.
Overall Responsibilities:
- Develop optimize and maintain data processing workflows using PySpark.
- Collaborate with crossfunctional teams to understand data requirements and provide data engineering solutions.
- Implement and support data pipelines for largescale data processing.
- Ensure the reliability efficiency and performance of data processing systems.
- Participate in code reviews and ensure adherence to best practices and standards.
Technical Skills:
Categorywise:
PySpark and Spark Development:
- Proficiency in PySpark and Spark for data processing.
- Experience with Spark modules and optimization techniques.
Unix and HDFS:
- Basic experience with Unix operating systems.
- Knowledge of Hadoop Distributed File System (HDFS).
Libraries and Tools:
- Familiarity with PyArrow and other related libraries.
Databases and SQL:
- Working knowledge of SQL and database management.
- Ability to write complex SQL queries for data extraction and manipulation.
Experience:
- Minimum of 6 years of professional experience in data engineering.
- Proven experience with PySpark and Spark development particularly for data processing.
- Experience working with Unix HDFS and associated libraries.
- Demonstrated experience with SQL and database management.
DaytoDay Activities:
- Develop and maintain data processing workflows using PySpark.
- Optimize Spark jobs for performance and efficiency.
- Collaborate with data engineers data scientists and other stakeholders to understand data requirements.
- Deploy and monitor data pipelines in a Hadoop and Spark environment.
- Perform data extraction transformation and loading (ETL) tasks.
- Troubleshoot and resolve issues related to data processing and pipeline failures.
- Participate in code reviews and contribute to the improvement of coding standards and practices.
Qualifications:
- Bachelors degree in Computer Science Information Technology or a related field.
- Relevant certifications in data engineering or big data technologies are a plus.
Soft Skills:
- Excellent communication and interpersonal skills.
- Strong problemsolving and analytical skills.
- Ability to work effectively in a fastpaced dynamic environment.
- Attention to detail and a commitment to delivering highquality work.
- Ability to prioritize tasks and manage time effectively.
- Collaborative and teamoriented mindset.
SYNECHRONS DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.
All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.
Required Experience:
Senior IC
Employment Type
Full-Time
