Big Data Lead Engineer — Hadoop, Spark & GCP (BigQuery/DataProc) with Java & Python Expertise

Job Summary

Synechron is seeking an experienced Big Data Lead Engineer to drive the design development and management of large-scale data solutions. This full-time leadership role involves guiding a team of data engineers and architects to deliver scalable efficient and innovative big data initiatives aligned with organizational and business objectives. The role aims to enable data-driven decision-making optimize data processing workflows and stay ahead of emerging industry trends to enhance the companys technological capabilities.

Software Requirements

Candidates must possess strong proficiency with big data technologies such as Hadoop Spark and NoSQL databases with proven experience deploying and managing these tools in enterprise environments. Familiarity with data warehousing ETL processes and data visualization tools is required. Hands-on experience with cloud platforms specifically Google Cloud Platform (GCP) is essential including managing cloud-based data processing tools like DataProc and BigQuery.

Preferred skills include experience with data pipeline orchestration tools automation frameworks and advanced analytics solutions. Knowledge of scripting languages such as Python shell scripting and familiarity with security and data governance practices will be advantageous.

Overall Responsibilities

• Lead a team of big data developers and architects to deliver high-quality data solutions providing technical guidance mentorship and support.

• Develop and execute a comprehensive big data strategy that supports business goals ensuring alignment with organizational priorities.

• Oversee project delivery ensuring timely within-budget execution that meets or exceeds stakeholder expectations.

• Design implement and optimize scalable big data architectures leveraging Hadoop Spark and NoSQL technologies.

• Collaborate with cross-functional teams including product technology and marketing to gather requirements and translate them into technical solutions.

• Stay current with evolving big data trends and emerging technologies assessing their applicability to improve existing workflows.

• Manage multiple projects balancing stakeholder needs with team capacity and prioritize efforts to maximize impact.

• Foster a culture of continuous learning professional growth and innovation within the team.

Technical Skills (By Category)

Programming Languages:
Required: Proficiency in Java and Python for developing applications workflows and automations.
Preferred: Knowledge of scripting languages such as Shell or Perl to support automation and deployment tasks.

Databases/Data Management:
Extensive experience with Hadoop ecosystem components (HDFS Hive Spark) and NoSQL databases like Cassandra or MongoDB. Strong capabilities with cloud data warehouses such as BigQuery and Cloud SQL.

Cloud Technologies:
Deep understanding of Google Cloud Platform (GCP) including DataProc BigQuery Cloud Storage and related cloud services for data management and processing.

Frameworks and Libraries:
Hands-on experience with Apache Spark Hive and other big data processing libraries; familiarity with data serialization formats like Parquet or Avro.

Development Tools and Methodologies:
Expertise in version control tools such as Git Agile development practices and automated CI/CD pipelines to promote efficient workflow management.

Security Protocols:
Knowledge of data security access control and compliance standards relevant to cloud data environments to ensure data protection.

Experience Requirements

• Minimum of 7 years of experience in big data and related software development roles.

• Real-world leadership experience managing teams of data engineers and architects for at least 5 years.

• Proven track record delivering large-scale big data projects within enterprise or consulting environments.

• Strong background in designing scalable architectures with Hadoop Spark and NoSQL databases.

• Industry experience in finance healthcare retail or technology sectors is preferred but not mandatory.

• Alternative candidates may demonstrate extensive hands-on experience in data engineering cloud data migrations or enterprise analytics projects.

Day-to-Day Activities

• Lead design and implementation of big data solutions including architecture development and data pipeline creation.

• Mentor team members through code reviews technical mentoring and knowledge-sharing sessions.

• Collaborate with stakeholders to clarify requirements define technical specifications and ensure solutions meet business needs.

• Monitor project progress manage workload and address technical challenges promptly.

• Ensure data solutions are optimized for performance reliability and security.

• Stay updated on industry trends and evaluate new tools or processes that could improve project outcomes.

• Coordinate across multidisciplinary teams to achieve project milestones and deliverables.

• Document technical architecture processes and best practices for team capacity building and future reference.

Qualifications

• Bachelors or Masters degree in Computer Science Data Science Engineering or a related technical discipline.

• Certifications in big data technologies (e.g. Cloudera Certified Professional Databricks or GCP certifications) are preferred.

• Proven experience designing and delivering large enterprise data solutions.

• Strong commitment to continuous learning and professional development.

• Excellent problem-solving analytical and troubleshooting skills.

Professional Competencies

• Critical thinking to analyze complex data challenges and develop innovative solutions.

• Leadership qualities to inspire mentor and develop technical talent.

• Effective communication skills capable of explaining complicated technical concepts to non-technical stakeholders.

• Strong collaboration and stakeholder management abilities.

• Adaptability to rapidly evolving technological trends and project demands.

• Ability to manage multiple priorities and deliver results within deadlines.

• Forward-thinking mindset focused on sustainable scalable and efficient analytics solutions.

SYNECHRONS DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.

All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.

Candidate Application Notice