Databricks Pyspark Developer

Chennai - India

Monthly Salary: Not Disclosed

Posted on: 7 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Databricks/ Pyspark Developer

Location: Remote

Job Summary: Looking for an offshore Senior Developer (for PMA and supporting EID/CJP) who has experience in Databricks/PySpark is willing to learn new technologies if needed and is able to work with team. The developer will be mainly focusing on implementing the Delete Act changes needed for PMA . Along with that he will support the optimizations/enhancement for EID/CJP.

Essential Job Functions:

Design and development of data ingestion pipelines (Databricks background preferred).
Performance tune and optimize the databricks jobs.
Evaluates new features and refractors existing code.
Develop and integrate software applications using suitable development methodologies and standards applying standard architectural patterns taking into account critical performance characteristics and security measures.
Collaborate with Business Analysts Technical Manager Architects and Senior Developers to establish the physical application framework (e.g. libraries modules execution environments).
Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary.
Works on best practices and documenting the process code merges and releases (Bitbucket).
Works with architect and manager on designs and best practices. Responsible for design of application considering the cost and best practices.
Good data analysis skills.
Must be willing to self learn new technologies become SMEs and develop high quality code in a fast paced environment.
Mentor junior developers and be hands on in development work.
Work with QA and automation team. Must have attention to detail to cover all scenarios for testing.
Handles and supports current production applications.
Performs code reviews and discussion on the changes with Technology Manager.

Minimum Qualifications and Job Requirements:

Must be a team player.
Must have at least 5 years of IT development experience.
Must have strong analytical and problem-solving skills.
Must have experience in designing solutions performing code reviews mentoring junior engineers.
Must have strong SQL and backend experience and working on data driven projects.

Must have the following experience: Python/PySpark SQL Databricks SCALA SQL Spark/Spark Streaming Big Data Tool Set Linux Kafka

Nice to have: Azure Data Factory

Required Experience:

Job Title: Databricks/ Pyspark DeveloperLocation: RemoteJob Summary: Looking for an offshore Senior Developer (for PMA and supporting EID/CJP) who has experience in Databricks/PySpark is willing to learn new technologies if needed and is able to work with team. The developer will be mainly focusing ...

Job Title: Databricks/ Pyspark Developer

Design and development of data ingestion pipelines (Databricks background preferred).
Performance tune and optimize the databricks jobs.
Evaluates new features and refractors existing code.
Develop and integrate software applications using suitable development methodologies and standards applying standard architectural patterns taking into account critical performance characteristics and security measures.
Collaborate with Business Analysts Technical Manager Architects and Senior Developers to establish the physical application framework (e.g. libraries modules execution environments).
Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary.
Works on best practices and documenting the process code merges and releases (Bitbucket).
Works with architect and manager on designs and best practices. Responsible for design of application considering the cost and best practices.
Good data analysis skills.
Must be willing to self learn new technologies become SMEs and develop high quality code in a fast paced environment.
Mentor junior developers and be hands on in development work.
Work with QA and automation team. Must have attention to detail to cover all scenarios for testing.
Handles and supports current production applications.
Performs code reviews and discussion on the changes with Technology Manager.

Minimum Qualifications and Job Requirements:

Must be a team player.
Must have at least 5 years of IT development experience.
Must have strong analytical and problem-solving skills.
Must have experience in designing solutions performing code reviews mentoring junior engineers.
Must have strong SQL and backend experience and working on data driven projects.

Must have the following experience: Python/PySpark SQL Databricks SCALA SQL Spark/Spark Streaming Big Data Tool Set Linux Kafka

Nice to have: Azure Data Factory

Required Experience:

Key Skills

Apply Now

About Company

Datamaxis

Job Summary: Able to provide guidance in all areas relating to information security in order to align and establish information security and strategy with business requirements. Primary Job Responsibilities: Cloud Security and/or Experience is preferred Automation, Scripting, Powe ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click