Senior Data Platform Engineer

Location - US Remote. Candidate to be based out of US.

KEY RESPONSIBILITIES

Database & System Architecture

• Design and implement robust scalable and secure database architectures for transactional and analytical workloads.
• Define partitioning strategies indexing and normalization for optimal performance and maintainability.

Performance Tuning & Optimization

• Conduct advanced query optimization and execution plan analysis.
• Implement proactive monitoring and troubleshooting for high-volume mission-critical systems.

Cloud & Distributed Systems

Azure Service Fabric:

• Deploy and manage microservices-based applications on Azure Service Fabric clusters.
• Ensure high availability fault tolerance and scalability of distributed services.
• Optimize communication patterns and stateful/stateless service design for data-intensive workloads.

Modern Data Engineering & Analytics

Azure Data Factory (ADF):

• Design and orchestrate complex ETL/ELT pipelines for batch and streaming data.
• Integrate diverse data sources into centralized data platforms.

Azure Databricks & Spark:

• Build and optimize big data processing workflows using Apache Spark on Databricks.
• Implement Delta Lake for ACID transactions on large-scale data lakes.
• Develop scalable solutions for machine learning and advanced analytics.

Delta Tables:

• Manage versioned data storage for reliability and reproducibility.
• Optimize schema evolution and data compaction strategies.

Power BI (PBI):

• Collaborate with BI teams to design semantic models and optimize data for reporting.
• Ensure data pipelines deliver clean consistent and performant datasets for visualization.

Collaboration & Leadership

• Work closely with architects DevOps and business stakeholders to align technical solutions with strategic objectives.
• Provide mentorship and technical guidance to junior engineers and developers.

REQUIRED SKILLS & QUALIFICATIONS

Database Mastery

• Advanced SQL query optimization partitioning and transaction handling.

Performance Tuning

• Proven ability to diagnose and resolve complex performance bottlenecks.

Azure Expertise

• Hands-on experience with Azure Service Fabric Azure Data Factory Azure Databricks and Azure Storage.

Big Data & Analytics

• Strong knowledge of Spark Delta Lake and distributed data processing.

Data Visualization

• Familiarity with Power BI and data modeling best practices.

Troubleshooting & Optimization

• Ability to resolve issues across multi-tiered systems under pressure.

Cross-Functional Collaboration

• Excellent communication and stakeholder management skills.

PREFERRED QUALIFICATIONS

• Experience with CI/CD pipelines for data solutions.
• Familiarity with data governance security and compliance frameworks.
• Exposure to machine learning workflows and real-time streaming architectures.

Required Experience:

Senior IC