Lead Sr. Pyspark Data Engineer

Gurgaon - India

Salary: Not Disclosed

Experience Required: 2-5years

Posted on: 9 hours ago

Vacancies: 1 Vacancy

Job Summary

AWS Cloud Data Engineer

Location Pune/HYD/Gurgaon

Job purpose:

This strategic role that -

Demonstrate lead level capabilities in conceptual analysis problem solving and data analysis. Demonstrate ability to take ownership or initiatives or projects with the ability to lead and facilitate business requirements and harness data to meet the requirements.

Must be able to collaborate with subject matter experts business analysts data analysts and other data stakeholders for data integration and alignment.

Provides support in the use of enterprise data including guidance in understanding and exploitation of the underlying data and business models and identifying optimal data sourcing and mapping.

Assists in developing and publishing data management standards policies and procedures including the management and maintenance of related tools and environments.

Create and review standard data deliverables ensuring adherence to defined corporate standards and definitions.

Demonstrate ability to understand produce and extend large and complex data models for operational and dimensional reporting data stores.

Experience coordinating information requirements of multiple departments including assessment of usefulness reliability and cleanliness of data from several sources.
* Ability to effectively present complex technical information to non-technical audiences and to senior decision-makers

Expertise and Experience:

4 8 years of experience in established data technologies viz. Data ingestion ETL and data warehouse CDC and Event Streaming technologies.

Strong hands on experience in AWS cloud services for data management and governance.

Develop pipelines to ingest data from different sources such as Database delimited files etc do complex transformation and loading data into the S3.

develop Glue mapping and create Glue data catalogue.

Hands on experience in design and implementation of data pipelines ETL and data warehouse technologies following industry standards.

Experienced or conversant in emerging tools like parquet or iceberg and unstructured data. Must understand them sufficiently to guide the organization in understanding and adopting them

Attitude/Behavior:

Strong sense of ownership of design solution

Advocate for the users perspective on the Agile team and promote the importance of design

Embraces a culture of trust and complete transparency

Open to learning new ideas outside scope or knowledge skillset

Ability to identify innovative business solutions and implement

Demonstrate success working in a team-based environment.

Information systems project management or leadership experience preferred.

Requires strong decision-making organizational planning and problem solving skills.

Excellent analytical skills required.

Strong facilitation and negotiation skills required.

Strong communication skills required; technical and business writing skills are required.

Strong interpersonal skills with competence in interfacing with business users required.

Ability to summarize and present design alternatives.

Ability to influence and sell soundness of design and deliverables.

Skills & Education:

Strong communication skills with ability to work with business and Technology stakeholders

Utilizes team collaboration to create innovative solutions efficiently

Comfortable working with quick turnaround times and deadlines

B.E/ from a reputed organization

Requirements

4 8 years of experience in established data technologies viz. Data ingestion ETL and data warehouse CDC and Event Streaming technologies.

Strong hands on experience in AWS cloud services for data management and governance.

Develop pipelines to ingest data from different sources such as Database delimited files etc do complex transformation and loading data into the S3.

develop Glue mapping and create Glue data catalogue.

Hands on experience in design and implementation of data pipelines ETL and data warehouse technologies following industry standards.

Experienced or conversant in emerging tools like parquet or iceberg and unstructured data. Must understand them sufficiently to guide the organization in understanding and adopting them

Required Skills:

Requirements Proficiency in Python programming. Advanced knowledge in mathematics and algorithm development. Experience in developing machine learning and deep learning models. Strong understanding of neural network architectures with emphasis on GenAI and LLMs. Skilled in data processing and visualization. Experienced in natural language processing. Knowledgeable in AI/ML deployment DevOps practices and cloud -depth understanding of AI security principles and practices.

AWS Cloud Data EngineerLocation Pune/HYD/GurgaonJob purpose: This strategic role that - Demonstrate lead level capabilities in conceptual analysis problem solving and data analysis. Demonstrate ability to take ownership or initiatives or projects with the ability to lead and facilitate busine...

AWS Cloud Data Engineer

Location Pune/HYD/Gurgaon

Job purpose:

This strategic role that -

Must be able to collaborate with subject matter experts business analysts data analysts and other data stakeholders for data integration and alignment.

Provides support in the use of enterprise data including guidance in understanding and exploitation of the underlying data and business models and identifying optimal data sourcing and mapping.

Assists in developing and publishing data management standards policies and procedures including the management and maintenance of related tools and environments.

Create and review standard data deliverables ensuring adherence to defined corporate standards and definitions.

Demonstrate ability to understand produce and extend large and complex data models for operational and dimensional reporting data stores.

Expertise and Experience:

4 8 years of experience in established data technologies viz. Data ingestion ETL and data warehouse CDC and Event Streaming technologies.

Strong hands on experience in AWS cloud services for data management and governance.

Develop pipelines to ingest data from different sources such as Database delimited files etc do complex transformation and loading data into the S3.

develop Glue mapping and create Glue data catalogue.

Hands on experience in design and implementation of data pipelines ETL and data warehouse technologies following industry standards.

Experienced or conversant in emerging tools like parquet or iceberg and unstructured data. Must understand them sufficiently to guide the organization in understanding and adopting them

Attitude/Behavior:

Strong sense of ownership of design solution

Advocate for the users perspective on the Agile team and promote the importance of design

Embraces a culture of trust and complete transparency

Open to learning new ideas outside scope or knowledge skillset

Ability to identify innovative business solutions and implement

Demonstrate success working in a team-based environment.

Information systems project management or leadership experience preferred.

Requires strong decision-making organizational planning and problem solving skills.

Excellent analytical skills required.

Strong facilitation and negotiation skills required.

Strong communication skills required; technical and business writing skills are required.

Strong interpersonal skills with competence in interfacing with business users required.

Ability to summarize and present design alternatives.

Ability to influence and sell soundness of design and deliverables.

Skills & Education:

Strong communication skills with ability to work with business and Technology stakeholders

Utilizes team collaboration to create innovative solutions efficiently

Comfortable working with quick turnaround times and deadlines

B.E/ from a reputed organization

Requirements

4 8 years of experience in established data technologies viz. Data ingestion ETL and data warehouse CDC and Event Streaming technologies.

Strong hands on experience in AWS cloud services for data management and governance.

Develop pipelines to ingest data from different sources such as Database delimited files etc do complex transformation and loading data into the S3.

develop Glue mapping and create Glue data catalogue.

Hands on experience in design and implementation of data pipelines ETL and data warehouse technologies following industry standards.

Experienced or conversant in emerging tools like parquet or iceberg and unstructured data. Must understand them sufficiently to guide the organization in understanding and adopting them

Required Skills: