ãRequire EnglishãData engineer (senior/ lead) - for someone living in Japan
ãRequire EnglishãData engineer (senior/ lead) - for someone living in Japan
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Description
One of the top insurance companies global working environment lets work together.
Job Title
Lead data engineer
Your Role and Responsibilities
As a Lead Data Engineer you will play a crucial role in the data engineering team taking a hands-on approach to developing key platform components using AWS and Databricks.
Your primary responsibility will be to deliver high-quality solutions while collaborating closely with fellow team members to ensure technical excellence and platform stability. Additionally you will be instrumental in successfully implementing business-driven solutions by:
Designing and delivering technical solutions that align with business needs within specified deadlines.
Coordinating with other technical teams to manage implementation dependencies effectively.
Providing accurate technical estimates and keeping stakeholders informed on progress.
Identifying and addressing potential technical challenges or risks early in the process.
Maintaining high-quality standards while meeting project timelines.
Contributing to solution design while balancing technical constraints with business objectives.
Key Responsibilities:
Platform Development
Design develop and maintain core data platform components leveraging AWS and Databricks.
Build and optimize data pipelines and transformation workflows.
Develop efficient data ingestion mechanisms for diverse data sources.
Establish and maintain automated testing and deployment pipelines.
Implement monitoring and alerting systems to ensure platform reliability.
Technical Implementation
Write clean scalable and well-structured code following best practices.
Extensive hands-on experience with Infrastructure as Code (IaC) particularly expertise in Terraform for infrastructure provisioning.
Implement security and compliance standards within platform components.
Enhance the performance of data processing workflows.
Develop technical documentation covering platform components and solution architecture.
Actively participate in code reviews and technical discussions.
Collaboration & Knowledge Sharing
Work closely with fellow Data Engineers and developers to ensure alignment in implementation patterns while challenging existing approaches when necessary.
Share technical expertise and industry best practices with team members.
Engage with cross-functional stakeholders to gather requirements and translate them into scalable and compliant solution architectures.
Take part in technical design discussions and contribute valuable implementation insights.
Maintain effective communication with project managers and business stakeholders.
Provide accurate development estimates and regular progress updates.
Proactively identify and escalate risks or roadblocks when necessary.
Experience and Qualifications
1. Experience & Technical Skills
Bachelors or Masters degree in Computer Science or a related technical discipline.
6 years of experience in data engineering and software development.
Strong expertise in cloud platforms (AWS preferred) and data processing frameworks.
Proficiency in Python (preferred) Scala or Java along with advanced SQL knowledge.
Hands-on experience with data processing frameworks (Spark preferred).
In-depth knowledge of modern SQL data warehouses (Databricks preferred Snowflake BigQuery) and columnar data formats (Parquet / Delta format ORC preferred).
Familiarity with modern data architectures (Lakehouse preferred).
Experience with DBT (Data Build Tool) and data modeling methodologies.
Strong background in CI/CD/CO practices and DevOps tools (JIRA Jenkins Terraform/AWS CDK for Infrastructure as Code).
Solid understanding of data modeling and data architecture principles.
Experience working with real-time data processing and streaming technologies.
2. Behavioral Competencies
Strong analytical and problem-solving abilities.
Keen attention to detail with a strong focus on quality.
Effective communication skills for collaboration across teams.
Self-motivated with a proactive work approach.
Ability to thrive in an agile development environment.
Commitment to continuous learning and professional growth.
3. Language Requirements
Business-level English proficiency required.
Japanese language skills are an advantage but not mandatory.
Benefits
Hybrid work
Work Location
Tokyo
Salary
10M
More details will be provided during the meeting.
Required Experience:
Senior IC
Employment Type
Full Time
