Data Engineer (Ab Initio, OzoneCH, Flink)

Role: Data Engineer (Ab Initio OzoneCH Flink)

Locations: Berkeley Heights NJ (5 Days Onsite)

Duration: Long term

Employment Type: W2 Contract / Fulltime

Job Description:

We are seeking a highly skilled resource to design and implement high-performance event-driven data pipelines ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE EME Conduct>It) to build stateful services that ingest filter and transform data from sources like Kafka or message queues pushing updates to dashboards or downstream databases in near-real-time.

Key Responsibilities:

• Create complex Ab Initio continuous flow graphs including stateful joins sliding time windows and aggregations.
• Implement event-driven data pipelines using Kafka MQ and file streams.
• Ensure the resilience of continuous flows including checkpointing and recovery to guarantee exactly-once processing.
• Apply advanced Ab Initio components (e.g. Reformat Rollup Join Partition) to ensure low-latency performance.
• Proactively monitor live production streams to ensure 24/7 reliability and troubleshooting data issues
• Develop ETL pipelines for batch and real-time data ingestion and transformation.
• Implement and ensure data validation data security integrity and compliance across big data platforms.
• Monitor and troubleshoot performance issues in large-scale clusters.
• Collaborate with data scientists analysts and application teams to deliver high-quality data solutions.
• Automate workflows and improve operational efficiency using scripting and orchestration tools.

Required Skills & Qualifications:

• Deep understanding of credit card process system
• Deep knowledge of GDE (Graphical Development Environment) EME (Enterprise Meta>Environment) Conduct>It and Continuous Flows.
• Understanding of Kafka message queues and real-time stateful services.
• Proficiency in Unix/Linux shell scripting SQL and database technologies (e.g. Oracle Teradata).
• Experience in Java Scala Python or Kafka is plus.
• Familiarity with Linux/Unix environments and shell scripting.
• Understanding of data security governance and compliance standards.

• Experience with cloud-based big data platforms
• Exposure to containerization (Docker Kubernetes) for big data workloads.
• Knowledge of CI/CD pipelines for data engineering projects.

Qualifications:

• Bachelors degree in computer science Software Engineering or a related field.
• Proficiency in business process modeling and documentation tools.
• Product implementation experience is preferred

Behavioral Skills:

• Good Communication skills
• 5 days Work from Office at Berkley Heights NJ
• Team Player
• Ability to work in a changing environment
• Strong problem solving and analytical skills
• Ability to work independently or within a team
• Manage day-to-day challenges and communicate developmental risks with the technical team

About Next Gen Software Solutions LLC:

Next Gen Software Solutions is a trusted provider of IT Staffing and consulting services dedicated to empowering businesses with cutting-edge technology solutions and exceptional talent. We specialize in delivering tailored IT consulting services innovative software solutions and connecting businesses with highly skilled IT professionals. Founded and led by a dedicated U.S. Army solider Next Gen Software Solutions is deeply rooted in the core values of integrity discipline commitment and experience-principles that guide every aspect of our operations.

Equal Employment Opportunity Statement:

Next Gen Software Solutions LLC is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all employees and applicants are treated respect and dignity. We do not discriminate based on race colour religion sex (including pregnancy sexual orientation or gender identity) national origin age genetic information veteran status or any other legally protected characteristic under applicable federal state or local laws.