Lead Data Engineer

Job Title: Lead Data Engineer

Experience: 7-10 Years

Location: Hyderabad India

Employment Type: Full-time

About NationsBenefits:

At NationsBenefits we are leading the transformation of the insurance industry by developing innovative benefits management solutions. We focus on modernizing complex back-office systems to create scalable secure and high-performing platforms that streamline operations for our clients. As part of our strategic growth we are focused on platform modernization transitioning legacy systems to modern cloud-native architectures that support the scalability reliability and high performance of core backoffice functions in the insurance domain.

Position Overview:

We are seeking a self-driven Lead Data Engineer with 710 years of experience to build and optimize scalable ETL/ELT pipelines using Azure Databricks PySpark and Delta Lake. The role involves working across scrum teams to develop data solutions ensure data governance with Unity Catalog and support real-time and batch processing. Strong problem-solving skills T-SQL expertise and hands-on experience with Azure cloud tools are essential. Healthcare domain knowledge is a plus.

Job Description:

Work with different scrum teams to develop all the quality database programming requirements of the sprint.

Experience in Azure cloud platforms like Advanced Python Programming Databricks Azure SQL Data factory (ADF) Data Lake Data storage SSIS.

Create and deploy scalable ETL/ELT pipelines with Azure Databricks by utilizing PySpark and SQL.

Create Delta Lake tables with ACID transactions and schema evolution to support real-time and batch processing.

Experience in Unity Catalog for centralized data governance access control and data lineage tracking

. Independently analyse solve and correct issues in real time providing problem resolution end-to-end.

Develop unit tests to be able to test them automatically.

Use SOLID development principles to maintain data integrity and cohesiveness.

Interact with product owner and business representatives to determine and satisfy needs.

Sense of ownership and pride in your performance and its impact on companys success.

Critical thinker and problem-solving skills.

Team player.

Good time-management skills.

Great interpersonal and communication skills.

Mandatory Qualifications:

7-10 years of experience as a Lead Data Engineer.

Self-driven with minimal supervision.

Proven experience with T-SQL programming Azure Databricks Spark (PySpark/Scala) Delta Lake Unity Catalog ADLS Gen2

Microsoft TFS Visual Studio Devops exposure.

Experience with cloud platforms such as Azure or any.

Analytical problem-solving mindset.

Preferred Qualifications

HealthCare domain knowledge