Python PypsarkPandas Engineer

Position Description:

We are looking for a skilled Python PySpark ETL Engineer with 3-6 years and 810 years of experience in building scalable data pipelines and processing large datasets. The ideal candidate should have hands-on expertise in PySpark strong Python programming skills and experience working with distributed data processing frameworks.

Your future duties and responsibilities:

Design develop and maintain ETL pipelines using Python and PySpark
Process large-scale structured and unstructured datasets efficiently
Work with big data technologies like Hadoop Spark and Hive
Optimize data workflows for performance and scalability
Collaborate with data analysts data scientists and stakeholders
Implement data quality checks and validation processes
Troubleshoot and resolve data pipeline issues
Ensure best practices in coding version control and deployment

Required qualifications to be successful in this role:

Strong proficiency in Python
Hands-on experience with PySpark and Apache Spark
Experience in ETL development and data pipeline design
Good knowledge of SQL and data warehousing concepts
Familiarity with Hadoop ecosystem (HDFS Hive etc.)
Experience with cloud platforms (AWS / Azure / GCP) is a plus
Understanding of data formats like Parquet ORC JSON
Knowledge of Airflow or other workflow orchestration tools

Skills:

• Pandas
• Python
• Spark SQL
• Testing
• Banking

What you can expect from us:

Together as owners lets turn meaningful insights into action.

Life at CGI is rooted in ownership teamwork respect and belonging. Here youll reach your full potential because

You are invited to be an owner from day 1 as we work together to bring our Dream to life. Thats why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our companys strategy and direction.

Your work creates value. Youll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas embrace new opportunities and benefit from expansive industry and technology expertise.

Youll shape your career by joining a company built to grow and last. Youll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

Come join our teamone of the largest IT and business consulting services firms in the world.