Senior Data Specialist R

Job Description Data Engineer (Databricks)

Company: Brillio Technologies
Role: Data Engineer Databricks
Experience: 510 Years
Location: India (Hybrid/3 days)
Employment Type: Full-time

About Brillio

Brillio is a leading digital transformation partner delivering innovative solutions across Data Cloud AI and Digital Engineering. We help global enterprises unlock value through cutting-edge technology and data-driven insights.

Role Overview

We are looking for a hands-on Data Engineer with strong Databricks expertise to design build and scale modern data platforms. The ideal candidate will have experience in PySpark Delta Lake and cloud-native data engineering and will work closely with analytics product and business teams to deliver high-performance data pipelines.

Key Responsibilities

• Design develop and maintain scalable ETL/ELT data pipelines using Databricks and PySpark
• Build and optimize data lakehouse architectures (BronzeSilverGold layers)
• Develop data ingestion frameworks from multiple sources (batch & real-time)
• Implement Delta Lake features including schema evolution ACID transactions and optimization
• Create and manage Databricks notebooks workflows and jobs for production pipelines
• Ensure data quality governance and lineage across pipelines
• Optimize Spark jobs for performance (partitioning caching tuning)
• Collaborate with cross-functional teams (Data Scientists Analysts Architects)
• Support production deployments monitoring and troubleshooting of data pipelines
• Contribute to migration of legacy data platforms to modern cloud-based architecture

Technical Skills (Must-Have)

• Strong experience in:
• Databricks (Workflows Notebooks Jobs)
• PySpark / Spark SQL
• SQL & Data Warehousing concepts
• Hands-on experience with:
• Delta Lake / Lakehouse architecture
• Data pipeline development & orchestration
• Good understanding of:
• Data modeling ETL/ELT processes SCD concepts

Cloud & Tools

• Experience in at least one:
• AWS (S3 Glue EMR) / Azure (ADF ADLS Synapse)
• Workflow tools:
• Airflow / Databricks Workflows
• Version control:
• Git / CI-CD pipelines

Good to Have

• Experience with:
• Streaming frameworks (Kafka / Spark Streaming)
• DBT / Snowflake / Redshift / Fabric
• Exposure to:
• Unity Catalog Autoloader Delta Live Tables

Experience Required

• 510 years of experience in Data Engineering
• Minimum 24 years of hands-on Databricks experience
• Proven experience building enterprise-scale data platforms and pipelines

Soft Skills

• Strong problem-solving and analytical skills
• Ability to work in a fast-paced agile environment
• Good communication and stakeholder management

Why Join Brillio

• Work on cutting-edge data & AI programs
• Opportunity to build large-scale data platforms
• Collaborative and innovation-driven culture