Job Description:

Technologies: Azure NLP RAG Semantic Matching Python

Role Summary

We are looking for a Data Scientist with expertise in Python Azure Cloud and NLP to build and enhance machine learning models at scale. The role includes embedding optimisation semantic matching LDA and RAG architectures dense and sparse retrieval pipelines and migration of cloud-native data pipelines to Azure Databricks.

Core Requirements

• Design and execute end-to-end machine learning pipelines including data extraction preprocessing feature engineering model development tuning and deployment.
• Develop machine learning pipelines using Azure Synapse Databricks and Snowflake.
• Build and deploy classification regression and clustering models.
• Develop and deploy proof-of-concept solutions for client use cases.
• Implement semantic matching and similarity search using cosine similarity dot-product scoring and bi-encoder/cross-encoder architectures (e.g. SBERT sentence-transformers).
• Build embedding models by fine-tuning pre-trained models and optimising embedding storage in vector databases such as Chroma DB FAISS and Azure AI Search.

Model Development & Optimisation

• Train and optimise models for new data providers with dynamic input handling.
• Improve LDA model performance for large-scale topic modelling.
• Implement hybrid semantic search by combining dense and sparse retrieval methods.
• Optimise RAG architectures and retrieval QA systems for chatbot and recommendation performance.
• Enable semantic query understanding using intent classification and query expansion techniques.

Forecasting & NLP

• Develop forecasting models for marketing demand prediction and trend analysis.
• Apply NLP-based forecasting techniques using sentiment and external data.
• Use semantic similarity for audience intelligence including zero-shot and few-shot classification techniques.

Data Pipeline & Cloud Migration

• Migrate data pipelines from Azure Synapse to Azure Databricks and retrain models accordingly.
• Optimise embedding storage and retrieval within Azure AI Search.
• Perform vector index tuning including HNSW optimisation and ANN benchmarking for production systems.

Required Skills & Tools

Python Azure Databricks Azure ML Azure Synapse Azure Blob Storage Scikit-learn NumPy Pandas Hugging Face sentence-transformers FAISS Chroma DB Azure AI Search LangChain TensorFlow PyTorch Statsmodels Azure OpenAI.

Location:

Brand:

Time Type:

Contract Type: