Senior PySpark Developer Engineer

Dallas, IA - USA

Monthly Salary: Not Disclosed

Posted on: 2 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Description:

Must-Have Technical Skills: PySpark SQL and UNIX

PySpark Expertise-

Strong hands-on experience with PySpark architecture understanding

Clear understanding of how PySpark jobs execute in the background

Ability to read write analyze and enhance PySpark scripts and configurations

SQL Proficiency:

Strong SQL skills

Experience working with SQL as part of data processing and PySpark job execution

Unix Shell Scripting:

Should have good understanding of Unix shell scripting.

Ability to understand backend Unix scripts supporting PySpark workflows

Experience in reading analyzing writing and modifying shell scripts for PySpark integration

ETL Knowledge:

Hands-on experience with ETL processes using any programming language or framework

Ability to understand and work with complementary ETL workflows alongside PySpark

Job Scheduling & Automation

Experience with at least one enterprise scheduling/automation tool such as: Autosys/ Control M/ Airflow/ or similar tools used in production environments

Nice-to-Have Skills

Experience working with finance domain applications

Familiarity with related frameworks or tools that enhance PySpark-based workflows

Job Description: Must-Have Technical Skills: PySpark SQL and UNIX PySpark Expertise- Strong hands-on experience with PySpark architecture understanding Clear understanding of how PySpark jobs execute in the background Ability to read write analyze and enhance PySpark scripts and configurations ...