Pyspark

Summary: As a Custom Software Engineer you will develop custom software solutions to design code and enhance components across systems or applications. Your typical day will involve collaborating with cross-functional teams to understand business requirements utilizing modern frameworks and agile practices to deliver scalable and high-performing solutions tailored to specific business needs. You will engage in coding testing and debugging activities while ensuring that the software meets quality standards and aligns with project timelines. Additionally you will participate in team meetings to discuss progress challenges and innovative ideas for improvement fostering a collaborative and productive work environment. Roles & Responsibilities: - Expected to be an SME.- Collaborate and manage the team to perform.- Responsible for team decisions.- Engage with multiple teams and contribute on key decisions.- Provide solutions to problems for their immediate team and across multiple teams.- Mentor junior team members to enhance their skills and knowledge.- Continuously evaluate and improve software development processes to increase efficiency.- Strong problem-solving skills and ability to work in a fast-paced environment Professional & Technical Skills: - Must To Have Skills: Proficiency in PySpark.- Strong hands-on experience with PySpark including optimization debugging and monitoring.- Strong understanding of data processing frameworks and distributed computing.- Good knowledge of AWS services: EMR S3 IAM and basic understanding of SageMaker.- Experience with modern software development methodologies particularly Agile.- Experience developing and managing Airflow workflows (DAGs).- Working knowledge of SQL for data analysis and validation.- Familiarity with cloud platforms and services for deploying applications.- Ability to troubleshoot and optimize performance issues in software applications. Additional Information: - The candidate should have minimum 5 years of experience in PySpark.- This position is based at our Gurugram office.- A 15 years full time education is required.