Senior Fullstack Data & AI Search Engineer100 Remote

Job Title -Senior Fullstack Data & AI Search Engineer
ExperienceRequired -8 Years
Timezone - Approx 1:00 PM / 2:00 PM and 10:00 PM / 11:00 PM IST (CET Time Zone)
WorkMode -Remote

Profile:Senior Data & AI Search Engineer with hands-on expertise inRAG pipelines and agentic AI workflows
Primary Focus:Elasticsearch RAG agentic AI workflows
Experience:Senior (8 years total; 6 years in enterprise search / RAG / LLM applications)

Role Overview:
We are looking for a hands-on Data & AI Search Engineer to design and deliver a production-grade AI-augmented enterprise search capability for a large international organisation. The engagement covers the full pipeline from raw data ingestion through to AI-generated grounded answers surfaced via a conversational or search interface.
The right candidate combinesdeep Elasticsearch engineeringwithpractical experience building Retrieval-Augmented
Generation (RAG) pipelinesandagentic AI workflows. This is an individual contributor role with direct impact on a critical knowledge management platform.

Key Responsibilities
1. Data Engineering and Ingestion
Design and build scalable ingestion pipelines and connectors from enterprise sources including SharePoint Liferay web crawls Data Lakes and corporate systems into Elasticsearch or equivalent search indexes.
Support batch incremental and near-real-time indexing; implement change tracking version management source provenance access permission mapping and deletion event handling to keep the index accurate.
Build document conversion pipelines for PDF Word Excel PowerPoint HTML email and scanned content; convert to structured Markdown and vector embeddings using tools such as Marker Docling or equivalent frameworks.
Design semantic chunking strategies (chunk size overlap section-aware splitting heading preservation table handling) and implement metadata extraction enrichment and deduplication during ingestion.

2. Retrieval and Search
Develop hybrid search capabilities combining BM25 keyword search semantic vector search metadata filtering and contextual retrieval.
Build re-ranking pipelines using embedding models cross-encoders or custom ranking logic to improve result relevance.
Implement advanced retrieval techniques: query rewriting query expansion multi-query retrieval parent-child retrieval contextual document embeddings and contextual compression.
Enforce security controls so users retrieve only content they are authorised to access.

3. RAG Pipeline and Agentic Workflows
Design and build the end-to-end RAG pipeline connecting enterprise search to large language models for grounded answer generation.
Implement agentic workflows where the AI can invoke tools call enterprise APIs perform multi-step reasoning and refine searches iteratively to answer complex queries.
Engineer prompt orchestration patterns: system prompts retrieval prompts guardrails context assembly response formatting and fallback strategies for low-confidence or ambiguous queries.

Technical Requirements

Core Search Engineering
Deep hands-on Elasticsearch experience: query DSL BM25 tuning functionscore boosting and decay functions multi-field matching.
Index and data modelling: field type selection custom analyzers and tokenizers per content type (code prose structured records multimedia).
Cluster operations: shard strategy index sizing reindexing query latency tuning and cluster health management.
Search evaluation and relevance testing: building ground-truth benchmarks measuring precision/recall NDCG and iterating against them.
Experience with Elasticsearch OpenSearch Azure AI Search or equivalent enterprise search platforms.

Data and Ingestion Engineering
Proven experience building or configuring connectors for SharePoint Liferay databases and Azure Data Lake including incremental sync CDC rate limiting and API edge-case handling.
Proficiency in Python; experience with data processing frameworks and document conversion libraries.

AI and RAG Engineering
Hands-on experience with embedding models re-ranking models cross-encoders prompt engineering and response grounding techniques.
Experience with LLM orchestration frameworks:LangChain LlamaIndex Haystack or equivalent.
Practical experience with tool calling agentic workflows function calling and multi-step retrieval.
Experience integrating with commercial or open-source LLMs: Azure OpenAI OpenAI Anthropic Google Gemini Meta Llama Mistral or similar.

Frontend
Working knowledge of React or equivalent front-end technologies to support search UI integration (desirable not mandatory).

Qualifications and Experience
First-level university degree in Computer Science Computer Engineering Information Systems or a related discipline.
8 years of professional experience in software or data engineering.
Minimum 6 years of hands-on experience building enterprise search AI-powered search semantic search RAG or LLM-based applications.
Excellent written and verbal communication skills in English.

What This Engagement Offers
We have built a high-visibility knowledge management platform for a large international organisation.
End-to-end ownership across data engineering retrieval and GenAI layers.
Fully remote flexible working arrangement within agreed time zone coverage.
Potential for contract extension based on delivery and business need.

Required Skills:

ElasticSearchArtificial IntelligenceMicrosoftHTML