Data Engineer (Spark, Hadoop, OzoneCH)

Berkeley Heights, NJ - USA

Monthly Salary: Not Disclosed

Posted on: 6 hours ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Data Engineer (Spark Hadoop OzoneCH)

Location: Berkley Heights NJ (5 Days Onsite)

Type: Contract W2 only

Duration: Long term

Role Overview:

We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark Hadoop ecosystem and Apache Ozone. The ideal candidate will design develop and optimize large-scale data processing systems ensuring high performance scalability and reliability for enterprise-level applications.

Key Responsibilities:

Design and implement distributed data processing solutions using Apache Spark and Hadoop
Develop and maintain Spark applications for data transformation aggregation and ETL processes using Scala Java or Python
Utilize Apache Ozone for storing large-scale datasets ensuring efficient data access and management in a distributed environment
Manage and optimize HDFS and Apache Ozone Kafka for scalable and fault-tolerant storage.
Develop ETL pipelines for batch and real-time data ingestion and transformation.
Implement and ensure data validation data security integrity and compliance across big data platforms.
Monitor and troubleshoot performance issues in large-scale clusters.
Collaborate with data scientists analysts and application teams to deliver high-quality data solutions.
Automate workflows and improve operational efficiency using scripting and orchestration tools.

Required Skills & Qualifications:

Strong expertise in Apache Spark (Core SQL Streaming).
Hands-on experience with Hadoop ecosystem (HDFS YARN MapReduce).
Proficiency in Apache Ozone for object storage and integration with Hadoop.
Solid programming skills in Java Scala or Python.
Experience with Hive HBase and Kafka is a plus.
Knowledge of cluster management and resource optimization.
Familiarity with Linux/Unix environments and shell scripting.
Understanding of data security governance and compliance standards.

Experience with cloud-based big data platforms
Exposure to containerization (Docker Kubernetes) for big data workloads.
Knowledge of CI/CD pipelines for data engineering projects.

Behavioral Skills:

Good Communication skills
5 days Work from Office at Berkley Heights NJ
Team Player
Ability to work in a changing environment
Strong problem solving and analytical skills
Ability to work independently or within a team
Manage day-to-day challenges and communicate developmental risks with the technical team

Qualifications:

Bachelors degree in computer science Software Engineering or a related field.
Proficiency in business process modeling and documentation tools.
Product implementation experience is preferred

About Next Gen Software Solutions LLC:

Next Gen Software Solutions is a trusted provider of IT Staffing and consulting services dedicated to empowering businesses with cutting-edge technology solutions and exceptional talent. We specialize in delivering tailored IT consulting services innovative software solutions and connecting businesses with highly skilled IT professionals. Founded and led by a dedicated U.S. Army solider Next Gen Software Solutions is deeply rooted in the core values of integrity discipline commitment and experience-principles that guide every aspect of our operations.

Equal Employment Opportunity Statement:

Next Gen Software Solutions LLC is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all employees and applicants are treated respect and dignity. We do not discriminate based on race colour religion sex (including pregnancy sexual orientation or gender identity) national origin age genetic information veteran status or any other legally protected characteristic under applicable federal state or local laws.

Job Title: Data Engineer (Spark Hadoop OzoneCH) Location: Berkley Heights NJ (5 Days Onsite) Type: Contract W2 only Duration: Long term Role Overview: We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark Hadoop ecosystem and Apache Ozone. The ideal candidate w...