Big Data Engineer – Hadoop, PySpark, Apache Kafka

Charlotte, VT - USA

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Must Have Technical/Functional Skills

Primary Skill: Hadoop ecosystem (HDFS Hive Spark)PySparkPythonApache Kafka

Secondary: UI Angular.

Experience: Minimum 9 years

Roles & Responsibilities

Architectural Leadership:

Define end-to-end architecture for data platforms streaming systems and web applications.

Ensure alignment with enterprise standards security and compliance requirements.

Evaluate emerging technologies and recommend adoption strategies.

Data Engineering :

Design and implement data ingestion transformation and processing pipelines using Hadoop PySpark and related tools.

Optimize ETL workflows for large-scale datasets and real-time streaming.

Integrate Apache Kafka for event-driven architectures and messaging.

Application Development :

Build and maintain backend services using Python and microservices architecture.

Develop responsive dynamic front-end applications using Angular.

Implement RESTful APIs and ensure seamless integration between components.

Collaboration & Leadership:

Work closely with product owners business analysts and DevOps teams.

Mentor junior developers and data engineers.

Participate in agile ceremonies code reviews and design discussions.

Required Skills & Qualifications:

Technical Expertise:

Strong experience with Hadoop ecosystem (HDFS Hive Spark).

Proficiency in PySpark for distributed data processing.

Advanced programming skills in Python.

Hands-on experience with Apache Kafka for real-time streaming.

Frontend development using Angular (TypeScript HTML CSS).

Architectural Skills:

Expertise in designing scalable secure and high-performance systems.

Familiarity with microservices API design and cloud-native architectures.

Additional Skills:

Knowledge of CI/CD pipelines containerization (Docker/Kubernetes).

Exposure to cloud platforms (AWS Azure GCP).

Education:

Bachelors or Masters degree in Computer Science Engineering or related field.

Experience:

9 years in software development with at least 4 years in architecture and Big Data technologies.

Preferred Qualifications:

BFSI domain experience or large-scale enterprise systems.

Understanding of data governance security and compliance standards.

Soft Skills:

Strong analytical and problem-solving abilities.

Excellent communication and leadership skills.

Ability to thrive in a fast-paced agile environment.

TCS Employee Benefits Summary:

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health Dental & Vision Disability Planning & Insurance Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation Time Off Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance 401K Plan Performance Bonus College Fund Student Loan Refinancing.

Salary Range: $100000 - $120000 a year

Must Have Technical/Functional Skills Primary Skill: Hadoop ecosystem (HDFS Hive Spark)PySparkPythonApache Kafka Secondary: UI Angular. Experience: Minimum 9 years Roles & Responsibilities Architectural Leadership: Define end-to-end architecture for data platforms streaming systems a...