Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
- Data Engineering & Pipeline Development
- Build and maintain ETL/ELT pipelines using Azure Databricks and Microsoft Fabric.
- Implement medallion architecture (Bronze Silver Gold layers) to support data lifecycle and quality.
- Develop real-time and batch ingestion processes from IES Gateway and other source systems.
- Ensure data quality validation and transformation logic is consistently applied.
- Use Python Spark and SQL in Databricks and Fabric notebooks for data transformation.
- Delta Lake: Implementing Delta Lake for data versioning ACID transactions and schema enforcement.
- Integration with Azure Services: Integrating Databricks with other Azure services like Azure One Lake Azure ADLS Gen2 and Microsoft fabric.
- Collaborate with the Domain Owners to design dimensional and real-time data models.
- Create analytics-ready datasets for Power BI and other reporting tools.
- Standardize field naming conventions and schema definitions across datasets.
- Apply data classification and tagging based on DECALs data governance framework.
- Implement row-level security data masking and audit logging as per compliance requirements.
- Support integration with Microsoft Purview for lineage and metadata management.
- Dimensional modeling
- Real-time data modeling patterns
- Partner with BI developers to ensure data models are optimized for Power BI.
- Provide curated datasets that align with reporting requirements and business logic.
- Create BI dashboards and train users.
- Support CI/CD pipelines for data workflows using Azure DevOps.
- Assist in monitoring logging and performance tuning of data jobs and clusters.
- Bachelors degree in computer science Data Engineering or related field.
- 3 years of experience in data engineering or analytics engineering roles.
- Advanced SQL: Proficiency in advanced SQL techniques for data transformation querying and optimization.
- Hands-on experience with:
- Azure Databricks (Spark Delta Lake)
- Microsoft Fabric (Dataflows Pipelines OneLake)
- SQL and Python (Pandas PySpark)
- SQL Server 2019
- Familiarity with data modeling data governance and data security best practices.
- Strong understanding of ETL/ELT processes data quality and schema design.
- Experience with Power BI datasets and semantic modeling.
- Knowledge of Microsoft Purview Unity Catalog or similar governance tools.
- Exposure to real-time data processing and streaming architectures.
- Knowledge of federal/state compliance requirements for data handling
- Familiarity with Azure DevOps Terraform or CI/CD for data pipelines.
- Microsoft Fabric Analytics Engineer.
- 3 years of experience in data engineering or analytics engineering roles.
- Advanced SQL: Proficiency in advanced SQL techniques for data transformation querying and optimization.
- Azure Databricks (Spark Delta Lake)
- Microsoft Fabric (Dataflows Pipelines OneLake)
- SQL and Python (Pandas PySpark).
Similar Jobs
Dr. Job is an online platform that connects employers with skilled job seekers, facilitating the search for job opportunities and top talent. Established in 2015. Dr. Job Pro has emerged as the world premier job portal, attracting thousands of job seekers every day from all over the world.
Follow Dr.Job
Dr Job FZ LLC. 2025 © All Rights Reserved