Lead Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
Vacancies:
Job Summary
Job Description
Position
Lead Data Engineer
Department: Data & Analytics
Employment Type: Full-Time
Language Requirement:
English: Business Level
Japanese: Business Level preferred (not mandatory)
Company: Global Technology & Consulting Organization
Role Overview
The Lead Data Engineer will be responsible for architecting developing and sustaining scalable data platforms that address complex business requirements. This role involves close collaboration with stakeholders and project teams to transform business needs into high-performance reliable and maintainable data solutions.
This position is ideal for a hands-on technical leader who takes ownership of system design enjoys solving complex challenges and can guide large-scale data engineering initiatives from concept through delivery.
Key Responsibilities
Work closely with business users and project stakeholders to gather and clarify requirements
Translate functional and non-functional business needs into well-designed data architecture solutions
Design build and optimize scalable data pipelines using technologies such as Scala Spark or Java
Develop and maintain reusable frameworks for data ingestion validation transformation and normalization
Create automation scripts supporting testing CI/CD processes data migration and quality validation
Provide technical support for troubleshooting incident analysis and system recovery
Contribute to engineering best practices coding standards and comprehensive technical documentation
Required Experience and Skills
36 years of experience as a software developer or data engineer
Strong programming experience in Scala Java or Python
Solid understanding of distributed data processing frameworks such as Hadoop and Spark including performance optimization
Experience working with public cloud environments with preference for Microsoft Azure
Knowledge of Oracle Database Microsoft SQL Server Linux environments and foundational networking concepts
Proficiency in SQL shell scripting Git version control unit testing and CI/CD tools
Strong analytical thinking and the ability to learn new technologies quickly
Preferred Qualifications
Familiarity with RESTful APIs NoSQL databases microservices architecture and ETL methodologies
Practical experience with cloud-native services
Experience in streaming data processing and change data capture (CDC)
Exposure to business intelligence tools is advantageous
Personal Competencies
Highly detail-oriented with the ability to independently lead the design and implementation of complex systems
Capable of clearly communicating technical concepts to non-technical stakeholders
Strong execution focus with the ability to apply expertise across multiple domains
Proactive learner motivated by new technologies and challenging problem-solving scenarios
Collaborative team player with strong interpersonal communication skills
Company Overview
This organization is a global consulting and technology services firm that partners with enterprises to drive digital transformation and sustainable growth. With a broad international presence and decades of industry experience the company delivers comprehensive services across strategy design engineering cloud data and operational transformation.
The firm combines advanced capabilities in artificial intelligence cloud computing and data platforms with deep industry expertise to create measurable value for clients worldwide.
Required Experience:
IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
