Lead Big Data Engineer (Java, Spark, Cloud, and Data Pipeline)
Lead Big Data Engineer (Java, Spark, Cloud, and Data Pipeline)
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Summary
Synechron is seeking an experienced Lead Big Data Engineer with a strong expertise in Java and Big Data frameworks to define develop and optimize scalable data solutions. This role is pivotal in designing high-performance data pipelines ensuring efficient processing and storage of large datasets and enabling data-driven decision-making across the organization. The Lead Engineer will provide technical leadership collaborate across teams and implement best practices to support the companys data ecosystem in a modern cloud-enabled environment.
Purpose:
To architect and deliver reliable scalable and secure big data solutions aligned with organizational data strategies.
Value:
This role ensures the organizations data infrastructure can support advanced analytics machine learning and real-time data processing thereby facilitating business growth and operational excellence.
Software Requirements
Required:
- Proficiency with Java programming (Java 8 or above).
- Strong experience working with Apache Spark (batch and streaming) with proven performance tuning skills.
- Hands-on experience with big data tools such as Hadoop HBase Couchbase and Phoenix.
- Experience deploying and managing data pipelines with CI/CD tools like Jenkins Bitbucket GIT Docker and OpenShift.
- Knowledge of data modeling ETL workflows and data architecture design.
- Familiarity with Linux/Unix environments and scripting.
Preferred:
- Experience with cloud-based big data solutions (e.g. AWS EMR Azure HDInsight).
- Knowledge of additional data tools like Kafka Spark SQL or Delta Lake.
Overall Responsibilities
- Lead the development and optimization of large-scale data pipelines and archive solutions using Java and Spark.
- Design implement and maintain robust data infrastructure supporting high-volume data ingestion transformation and storage.
- Collaborate with data scientists analysts and cross-functional teams to gather requirements provide technical solutions and ensure data quality.
- Develop and enforce standards for code quality performance security and scalability.
- Oversee deployment automation and system monitoring to ensure high availability and performance.
- Conduct code reviews provide technical mentorship and guide best practices across the data engineering team.
- Keep abreast of emerging big data and cloud technologies recommending their integration to enhance data platform capabilities.
Strategic Objectives:
- Deliver high-performing scalable data platforms that underpin analytics and business intelligence initiatives.
- Enhance operational efficiency and data reliability through automation and best practices.
- Promote innovative use of emerging data technologies to unlock new insights.
Performance Outcomes:
- Stable high-performance data processing pipelines delivering data promptly and accurately.
- Reduced system downtime and optimized resource utilization.
- Effective technical guidance leading to a cohesive skilled data engineering team.
Technical Skills (By Category)
Programming Languages:
- Essential: Java (Java 8 or higher) for data pipeline development and system integration.
- Preferred: Knowledge of Python or Scala for additional scripting flexibility.
Databases & Data Management:
- Extensive experience working with big data frameworks such as Hadoop HBase Couchbase and Phoenix.
- Understanding of data warehousing data lakes and schema design.
Cloud Technologies:
- Hands-on experience deploying big data solutions on cloud platforms such as AWS Azure or GCP.
- Knowledge of cloud-native data services and security practices.
Frameworks & Libraries:
- Expertise with Apache Spark (batch and streaming) Spark SQL and related libraries.
- Familiarity with data serialization formats like Parquet Avro or ORC.
Development Tools & Methodologies:
- Proficiency in version control systems like GIT Jenkins and containerization tools such as Docker and OpenShift.
- Agile methodologies for iterative delivery and continuous integration.
Security & Data Governance Protocols:
- Knowledge of data security best practices access controls and privacy regulations relevant to big data.
Experience Requirements
- Minimum of 5 years as a Data Engineer or related role with 3 years in large-scale data pipeline development.
- Demonstrated experience designing building and maintaining enterprise-grade big data platforms.
- Proven track record of implementing big data solutions using Java and Spark in a production environment.
- Industry experience in finance healthcare retail or telecom sectors preferred.
- Experience leading technical teams and mentoring junior engineers.
Day-to-Day Activities
- Design develop and optimize data pipelines and storage solutions utilizing Java and Spark.
- Collaborate with data analysts data scientists and business users to understand requirements and deliver solutions.
- Execute continuous integration and deployment of data solutions to cloud environments.
- Monitor system performance troubleshoot issues and implement improvements proactively.
- Conduct code reviews and provide guidance to ensure adherence to best practices.
- Create and update technical documentation data models and operational procedures.
- Stay current with industry trends and incorporate new tools or techniques to improve the data platform.
Qualifications
- Bachelors or Masters degree in Computer Science Information Technology or equivalent.
- Professional certifications related to Big Data (e.g. Cloudera Hortonworks AWS Big Data Specialty) or Java development are advantageous.
- Proven ability to lead large scale data projects and drive technical excellence.
Professional Competencies
- Strong analytical and troubleshooting skills for complex data systems.
- Excellent communication skills for stakeholder engagement and documentation.
- Leadership ability with experience mentoring and guiding teams.
- Adaptability to rapidly evolving data technologies and organizational priorities.
- Innovative mindset to leverage emerging platforms and tools for business value.
- Effective time management to prioritize tasks and meet deadlines.
SYNECHRONS DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.
All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.
Employment Type
Full-Time
