Big Data Developer

Columbus, NE - USA

Monthly Salary: Not Disclosed

Posted on: 13 hours ago

Vacancies: 1 Vacancy

Job Summary

About the Role

The Big Data Developer will play a critical role on the Big Data engineering team designing and implementing large-scale data processing systems that power scientific research and innovation. The ideal candidate has hands-on experience building enterprise-grade data pipelines working with distributed systems and optimizing data processing workflows.

This is a long-term project (12 months) with high visibility cutting-edge tools and opportunities to influence technical direction.

What Youll Do

Data Pipeline Design & Development

Design build and deploy scalable data pipelines for ingesting processing transforming and storing high-volume datasets.
Implement streaming and batch-processing solutions using Hadoop Spark and cloud-based tools.

Data Architecture & Engineering

Develop and maintain data architecture and data flow models.
Ensure data reliability accuracy and integrity across all environments.
Support data warehousing strategies and best practices.

Data Quality Security & Compliance

Implement automated data validation error handling and monitoring.
Ensure compliance with internal security controls and regulatory standards.
Partner with governance teams to enforce data quality and security guidelines.

Cross-Functional Collaboration

Work closely with data scientists analysts product teams and application developers.
Translate business requirements into robust technical solutions.
Participate in Agile ceremonies and contribute to technical design discussions.

Performance Optimization

Tune Spark applications Hadoop jobs and distributed data systems for performance and cost efficiency.
Troubleshoot bottlenecks and implement improvements to system performance.

Technical Leadership

Provide mentorship to junior developers and contribute to coding standards best practices and technical documentation.

Required Skills & Qualifications

4 years of Big Data Development experience in Hadoop ecosystems
2 years of hands-on development with Apache Spark
Proficiency in Java Scala or Python
Strong understanding of distributed systems ETL data warehousing and data modeling concepts
Experience with large-scale datasets performance tuning and troubleshooting
Strong problem-solving communication and collaboration skills
Bachelors degree in Computer Science Engineering or related discipline

Preferred Skills

Experience working with AWS cloud services (EMR S3 Lambda Glue etc.)
Experience with Spark 3.x or 4.x
Exposure to Kubernetes Airflow or similar orchestration tools
Familiarity with CI/CD and DevOps automation for data engineering

Why This Opportunity Stands Out

Long-term project stability (12 months likely extension)
Ability to work on high-impact scientific and research-driven datasets
Hands-on cloud modernization (AWS) and next-generation big data tooling
Collaborative and innovative engineering culture

About the Role The Big Data Developer will play a critical role on the Big Data engineering team designing and implementing large-scale data processing systems that power scientific research and innovation. The ideal candidate has hands-on experience building enterprise-grade data pipelines workin...