Lead Data Engineer Fabric
Share
Job Location:
Monthly Salary:
Experience Required:
Posted on:
6 hours ago
Vacancies:
1 Vacancy
Job Summary
About The Company:
What makes iLinks offerings unique is the fact that we use pre-created frameworks designed to accelerate software development and implementation of business processes for our clients. iLink has over 60 frameworks (solution accelerators) both industry-specific and horizontal that can be easily customized and enhanced to meet your current business challenges.
Requirements
We are looking for a highly skilled Lead Data Engineer with strong expertise in Microsoft Fabric to design develop and manage modern data platforms. The ideal candidate will drive end-to-end data engineering solutions mentor the team and collaborate closely with business and technical stakeholders to deliver scalable and high-performance data pipelines.
Key Responsibilities
- Lead the design and implementation of end-to-end data solutions on Microsoft Fabric including Data Factory Lakehouse Warehouse and Data Engineering experiences.
- Build and optimize ETL/ELT pipelines using PySpark Dataflows Gen2 and Fabric pipelines.
- Architect and maintain Lakehouse and Delta Lake structures for large-scale data processing.
- Ensure strong data governance quality lineage and security across the ecosystem.
- Work closely with data architects analysts and business partners to understand requirements and convert them into technical solutions.
- Oversee performance tuning capacity planning cost management and platform optimization.
- Mentor junior data engineers and provide best practices for coding design and deployment.
- Implement CI/CD processes for Fabric workloads (Git integration workspace management DevOps).
- Troubleshoot data pipeline failures production issues and optimize workloads.
- Stay updated with Microsoft Fabric roadmap and bring innovative solutions into the architecture.
Required Skills & Experience
- 7 years of experience in Data Engineering with at least 2 years in Microsoft Fabric or modern data platforms.
- Strong hands-on experience in:
- PySpark / Spark SQL
- Fabric Data Engineering Lakehouse Warehouse
- Pipelines Dataflows Gen2 KQL Notebooks
- Delta Lake and Medallion architecture
- PySpark / Spark SQL
- Solid understanding of data modeling data warehousing and SQL performance tuning.
- Experience with Azure services: ADLS Synapse Data Factory Databricks (added advantage).
- Expertise in Git DevOps practices and production deployment workflows.
- Strong analytical/problem-solving skills and stakeholder management capabilities.
- Experience leading a team or owning end-to-end delivery of data solutions.
Good to Have
- Knowledge of Power BI integration with Fabric
- Experience with Real-time analytics / Event streaming
- Familiarity with Python development and API integrations
- Exposure to ML pipelines within Fabric
Benefits
- Competitive salaries
- Medical Insurance
- Employee Referral Bonuses
- Performance Based Bonuses
- Flexible Work Options & Fun Culture
- Robust Learning & Development Programs
- In-House Technology Training
Required Skills:
Requirements We are seeking a visionary AI Architect to lead the design development and implementation of cutting-edge AI solutions. The ideal candidate will have a strong background in machine learning data engineering cloud architecture and enterprise-grade solution design. You will collaborate with stakeholders to translate business challenges into scalable AI-driven systems. Key Responsibilities: Lead the design and architecture of AI/ML solutions across the organization. Collaborate with data scientists engineers product owners and business leaders to define AI use cases and deliverables. Define architecture standards and best practices for AI solution development and deployment. Ensure AI models are scalable robust and integrated with existing systems and data platforms. Oversee end-to-end MLOps processes including versioning monitoring and continuous integration/deployment (CI/CD). Evaluate and select AI tools frameworks and technologies aligned with enterprise needs. Stay current on AI/ML trends tools and regulations (including ethical AI practices). Provide technical leadership and mentorship to development teams. Prepare architecture documentation and presentations for stakeholders and leadership. Required Qualifications: Bachelors or Masters degree in Computer Science Data Science Artificial Intelligence or related field. Proven experience in architecting AI/ML solutions in cloud environments (AWS Azure or GCP). Deep understanding of machine learning algorithms NLP computer vision and deep learning frameworks (e.g. TensorFlow PyTorch Hugging Face). Strong proficiency in Python and relevant ML libraries. Solid experience with data engineering pipelines data lakes and APIs. Familiarity with MLOps platforms (e.g. MLflow Kubeflow SageMaker Azure ML). Strong knowledge of software architecture patterns and microservices. Excellent communication stakeholder management and documentation skills. Preferred Qualifications: AI/ML certification (e.g. Google Professional ML Engineer AWS Machine Learning Specialty). Experience working in regulated industries (healthcare finance etc.). Familiarity with Generative AI LLMs (like GPT) and Responsible AI principles. Experience in DevOps or Infrastructure as Code (e.g. Terraform Docker Kubernetes).
Required Education:
Any Professional Degree
Company Industry
IT Services and IT Consulting
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
