Senior Data Engineer MS Fabric

Roles & Responsibilities :

We are seeking a skilled Senior Data Engineer with expertise in Databricks to join our dynamic data team. The ideal candidate will design build and maintain scalable data pipelines and architectures to support our organizations data-driven initiatives and leverage Databricks to process large-scale datasets optimize data workflows enable advanced analytics and machine learning and integrate Power BI for data visualization and reporting.

Key Responsibilities

• 5-8 years of professional work experience in a relevant field

• Proficient in Microsoft Fabric platform Azure Databricks ADF Delta Lake SQL Data Warehouse Unity Catalog.

• Good Experience on Microsoft Dynamics 365Experience/ prior knowledge on semi structure data and Structured Streaming Azure synapse data lake data warehouse.

• Proficient in creating Azure Data Factory pipelines for ETL/ELT processing; copy activity custom Azure development etc.

• Good knowledge of SQL and Python for data manipulation transformation and analysis

• Understand business requirements to set functional specifications for reporting applications

Data Pipeline Development: Design develop and maintain robust scalable data pipelines using Databricks Apache Spark and other cloud-based technologies.

Data Integration: Ingest transform and integrate data from diverse sources including APIs databases streaming platforms and third-party systems into Databricks for analytics and reporting.

Power BI Integration: Develop and optimize data models and datasets in Databricks for use in Power BI ensuring efficient data connections and high-quality visualizations.

Performance Optimization: Optimize data workflows API calls and queries on Databricks and Power BI for performance cost-efficiency and scalability.

Data Modeling: Build and maintain data models to support business requirements ensuring data quality consistency and accessibility for analytics and reporting.

Cloud Integration: Implement data solutions on cloud platforms (e.g. AWS Azure GCP) integrated with Databricks Power BI and API ecosystems.

Security & Compliance: Implement data governance security and compliance best practices within Databricks Power BI and API environments.

Technical Skills:

• Proficiency in Databricks including Delta Lake Spark SQL.

• Strong programming skills in Python Scala or Java.

• Experience with Apache Spark for big data processing.

• Knowledge of SQL for querying and transforming data.

• Proficiency in Power BI for creating data models DAX queries and interactive dashboards.

Preferred Qualifications

• Databricks certification (e.g. Databricks Certified Data Engineer Associate/Professional).

• Experience with real-time data processing and streaming (e.g. Kafka Spark Streaming).

Additional Information :

Educational qualification:

BE/ or Equivalent Degree

Experience :

2-5 Years

Mandatory/requires Skills :
Strong hands-on expertise in Python (or Java) NLP RegEx SpaCy NLTK and DevOps

Preferred Skills :

Remote Work :

No

Employment Type :

Full-time