Lead Data Engineer

Role Purpose

Exp : 8yrs to 15 yrs

Location : Chennai

Np : Immediate to 30days

Required Skills

Databricks (Highest Priority)

Databricks

• PySpark
• Spark SQL
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Databricks Asset Bundles
• Lakehouse Architecture

Azure Platform

Azure Data Factory (ADF)

• ADLS Gen2
• Azure SQL
• Azure Security Patterns

Migration Experience

Synapse Pipelines

• Synapse Spark
• Synapse SQL Pools
• Synapse Notebooks
• Synapse Data Flows
• Modernization of legacy ETL workloads

Engineering

Advanced SQL

• Data Modeling
• GitLab CI/CD
• Performance Tuning
• Code Reviews

Preferred

Apache Airflow

• Snowflake
• Data Governance

Enterprise-scale Azure platform implementation

Areas of responsibility

Gathering of requirements-Assist in conducting workshops and discussions with client stakeholders to gather business process requirements.

Coding and Configuration-Develop and deliver code for assigned modules in alignment with client requirements ensuring proper integration with system components.

Testing and Trial Runs-Execute testing for assigned modules validate functionality across required interactions and participate in User Acceptance Testing (UAT) sessions.

Implementation-Participate in implementation of software application and its integration with other systems to ensure stability.

Maintain Documentation-Prepare detailed reports and documentation on project specifications activities and status while presenting information using flowcharts layouts diagrams code comments and clear well-structured code.

Stakeholder Collaboration-Collaborate with cross functional teams to understand requirements support development activities check system compatibility and ensure solutions meet expected standards.