Data & AI Engineer

We are seeking talented Data & AI Engineers to support the development of an enterprise-grade AI Assistant. This is a high-impact initiative at the intersection of modern data engineering and applied AI. The role focuses on building the data pipeline that transforms unstructured corporate documents into a searchable intelligent knowledge base powering a Large Language Model (LLM) as well as automating actions such as ticket creation and workflow integrations.

Tasks

Data Engineering

• Build and maintain scalable data pipelines using Databricks and Azure Data Lake Storage Gen2 (ADLS Gen2) ensuring reliability and performance across all pipeline stages.

Data Transformation

• Implement the Medallion Architecture (Raw Silver Gold layers) to clean normalise and structure technical documentation for downstream AI consumption.

AI Orchestration

• Design and develop the Reasoning Layer using frameworks such as LangChain or LlamaIndex to manage LLM logic prompt routing and tool orchestration.

Search Optimisation

• Manage and tune Vector Databases (e.g. Pinecone Weaviate or Azure AI Search) to enable fast and accurate semantic retrieval across the corporate knowledge base.

Integration & Automation

• Develop Azure Apps and Azure Functions to connect the AI system to external enterprise platforms such as Salesforce and SharePoint enabling end-to-end automation.

Requirements

• 34 years in Data Engineering or Software Development with a strong focus on cloud-based data processing.
• Strong proficiency in Databricks and Python; comfortable working across the full data engineering lifecycle.
• Hands-on experience with LLM orchestration frameworks: LangChain LlamaIndex or similar.
• Solid understanding of the Azure ecosystem Storage (ADLS Gen2) Identity (AAD) and Serverless (Azure Functions).
• Familiarity with modern AI coding assistants (GitHub Copilot Cursor OpenAI Codex) to accelerate development velocity.