Lead Data Engineer

Position: Lead Data Engineer

Location: San Francisco CA - Hybrid

Duration: 9Months with possible extension

Interviews: (4 Rounds): All the interviews within one week.

1. General Interview

2. Technical Interview

3. Project Interview

4. Client Interview

Main skills: SQL Databricks ADF DataStage (or other ETL tool) SSAS cubes Cognos/other Tableau/other ThoughtSpot

Job Description:

We are seeking a Lead Data Engineer to design implement and maintain our data infrastructure and business intelligence solutions. This role requires a technical expert who can architect scalable data solutions while collaborating across teams to drive business insights.

Required Experience:

8-10 years of software development and deployment experience

At least 7 years of hands-on experience with:

SQL

Databricks

Azure Data Factory (ADF)

DataStage (or other ETL tools)

SSAS cubes

BI tools (Cognos Tableau ThoughtSpot etc.)

Strong Database knowledge (SQL & No-SQL preferred)

Advanced SQL and Python skills

Experience working with APIs for data collection/ingestion

Key Responsibilities:

Technical Development:

Write SQL for processing raw data Kafka ingestions ADF pipelines data validation and QA

Lead design and implement innovative analytical solutions using Tableau SQL and Big Data technologies

Experience with storing joining filtering and analyzing data using SQL Spark Hive etc.

Data Architecture & Design:

Design comprehensive data architecture and data modeling strategies for BI systems

Create logical and physical data models using best practices to ensure high data quality and reduced redundancy

Collaborate with stakeholders to translate business requirements into robust scalable data models

Perform detailed analysis of business problems and technical environments to design quality solutions

Integration & Implementation:

Integrate data from various sources (databases warehouses external systems).

Develop strategies for ETL processes ensuring data accuracy consistency and availability.

Evaluate and select appropriate BI tools based on organizational needs.

Run POCs and implement BI software including visualization tools and analytics solutions.

Optimization & Governance:

Monitor and optimize BI system performance for efficient data retrieval processing and reporting

Identify and resolve performance bottlenecks through query tuning and data structure optimization

Establish data governance policies ensuring data quality consistency and security

Define access controls privacy measures and retention policies to meet regulatory requirements

Collaboration & Leadership:

Build partnerships with architects technical leads and stakeholders across functional groups.

Actively participate in code reviews and testing to ensure adherence to best practices.

Work with product and engineering teams to evaluate requirements and new features.

Build and foster a high-performance engineering culture.

Mentor team members and provide tools and motivation for success.

Additional Skills & Requirements:

Experience building data ingestion pipelines and data warehouse architecture

Ability to write design documentation source-to-target mapping documentation and manage Confluence pages

Experience converting business functionalities into technical Jira stories

Strong experience with data modeling design patterns and building scalable BI solutions

Knowledge of cloud platforms (Azure AWS or equivalent)

Experience analyzing large datasets to identify trends patterns and outliers

Regards

Bala Krishna

Talent Acquisition Specialist

NasTech Global Inc

Ph#:

Email:

CONFIDENTIAL AND PROPRIETARY. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient you are notified that disclosing copying distributing or taking any action in reliance on the contents of this information is strictly prohibited. NasTech Global Inc. All rights reserve.