Senior Data Scientist – AIML, Databricks & LLM
Job Summary
- Lead the development and delivery of functional and ministry-specific analytics to support evidence-based decision-making.
- Execute and monitor batch inference jobs using Databricks and other cloud/data environments.
- Troubleshoot inference issues investigate model anomalies document results and escalate issues when required.
- Assist with DSPy pipelines including loading and configuring custom models and configurations.
- Support evaluation runs using established/internal metrics and methodologies.
- Track analyze and summarize performance metrics from inference and experimentation activities.
- Prepare clean transform and validate structured and unstructured datasets including legal text datasets.
- Annotate model outputs to support error analysis benchmarking and model evaluation.
- Develop and maintain reproducible Python scripts notebooks and analytical workflows.
- Work with stakeholders to understand current analytics and reporting capabilities and identify future-state requirements and opportunities for improvement.
- Own or contribute to analytics initiatives from data preparation through reporting and dataset delivery.
- Develop statistical models and machine learning algorithms to model business scenarios and derive valid inferences.
- Design methods for capturing structuring transforming and processing data for analytical and machine learning purposes.
- Build reliable data models that provide accurate understandable and unbiased information.
- Work closely with Data Architects ETL Developers functional experts and business stakeholders to develop appropriate Business Intelligence and analytical solutions.
- Participate in solution documentation development testing implementation and end-user training.
- Communicate complex quantitative analysis clearly to both technical and non-technical audiences through effective visuals summaries and recommendations.
- Provide analytical interpretation advice and guidance to client groups and stakeholders on converting analytics into actionable and proactive insights.
- Facilitate decision-making manage expectations and provide consulting and relationship-management support.
- Hands-on experience with Microsoft Azure data tools/cloud technologies.
- Strong experience with Azure Databricks.
- Experience with MLflow and machine learning lifecycle/model experimentation environments.
- Strong proficiency in Python.
- Experience with Python data science libraries such as pandas scikit-learn PyTorch or equivalent.
- Exposure to LLM frameworks including DSPy LangChain Hugging Face or equivalent.
- Experience working with data preparation transformation analysis experimentation and/or machine learning workflows.
- Ability to manipulate and analyze complex high-volume data from structured and unstructured sources.
- Statistical analysis and modelling.
- Data mining and machine learning.
- Machine learning algorithms and model evaluation.
- Natural language processing and related analytical disciplines.
- Data extraction transformation and loading concepts.
- Complex query development and query languages.
- Data modelling and database concepts.
- Data management and database architecture.
- Business and financial analysis.
- Information visualization and analytical reporting.
- Mathematics and statistics.
- Strong investigative logical analytical and problem-solving abilities.
- Understanding of emerging Business Intelligence Data Science AI and ML trends.
- LLM inference workflows.
- DSPy pipelines and evaluations.
- Other LLM frameworks such as LangChain or Hugging Face.
- Databricks batch inference and data processing.
- MLflow for experiment/model tracking and ML lifecycle activities.
- Model performance evaluation and metric tracking.
- Dataset annotation and error analysis.
- Reproducible notebooks and scripts.
- Model experimentation and benchmarking.
- R
- PowerPivot
- MATLAB
- SPSS
- SAS
- Microsoft Excel
- Microsoft Access
- VBA
- Relational and multidimensional data stores
- Business Intelligence and visualization platforms
- Background in Computer Science Data Science Artificial Intelligence Machine Learning or a closely related discipline.
- Strong analytical problem-solving decision-making and investigative skills.
- Excellent verbal and written communication skills.
- Strong interpersonal collaboration and teamwork abilities.
- Excellent consulting and relationship-management skills.
- Demonstrated ability to elicit requirements and develop or advise on analytical options and solutions.
- Ability to communicate effectively with both technical and non-technical audiences.
- Strong organizational skills and attention to detail particularly when tracking experiments model performance and analytical deliverables.
- Ability to work effectively with multidisciplinary teams stakeholders architects developers and business experts.
- Technical Skills 50%
- LLM Frameworks 25%
- Databricks / MLflow 25%
Required Skills:
Mandatory Technical Skills Candidates must have: Hands-on experience with Microsoft Azure data tools/cloud technologies. Strong experience with Azure Databricks. Experience with MLflow and machine learning lifecycle/model experimentation environments. Strong proficiency in Python. Experience with Python data science libraries such as pandas scikit-learn PyTorch or equivalent. Exposure to LLM frameworks including DSPy LangChain Hugging Face or equivalent. Experience working with data preparation transformation analysis experimentation and/or machine learning workflows. Ability to manipulate and analyze complex high-volume data from structured and unstructured sources. Data Science & Analytics Skills Statistical analysis and modelling. Data mining and machine learning. Machine learning algorithms and model evaluation. Natural language processing and related analytical disciplines. Data extraction transformation and loading concepts. Complex query development and query languages. Data modelling and database concepts. Data management and database architecture. Business and financial analysis. Information visualization and analytical reporting. Mathematics and statistics. Strong investigative logical analytical and problem-solving abilities. Understanding of emerging Business Intelligence Data Science AI and ML trends. LLM Databricks & Experimentation The successful candidate should be comfortable working within modern AI/ML environments and supporting: LLM inference workflows. DSPy pipelines and evaluations. Other LLM frameworks such as LangChain or Hugging Face. Databricks batch inference and data processing. MLflow for experiment/model tracking and ML lifecycle activities. Model performance evaluation and metric tracking. Dataset annotation and error analysis. Reproducible notebooks and scripts. Model experimentation and benchmarking. Additional Tools & Technologies Experience with any of the following is considered valuable: R PowerPivot MATLAB SPSS SAS Microsoft Excel Microsoft Access VBA Relational and multidimensional data stores Business Intelligence and visualization platforms General Qualifications Background in Computer Science Data Science Artificial Intelligence Machine Learning or a closely related discipline. Strong analytical problem-solving decision-making and investigative skills. Excellent verbal and written communication skills. Strong interpersonal collaboration and teamwork abilities. Excellent consulting and relationship-management skills. Demonstrated ability to elicit requirements and develop or advise on analytical options and solutions. Ability to communicate effectively with both technical and non-technical audiences. Strong organizational skills and attention to detail particularly when tracking experiments model performance and analytical deliverables. Ability to work effectively with multidisciplinary teams stakeholders architects developers and business experts. Evaluation Weighting Technical Skills 50% LLM Frameworks 25% Databricks / MLflow 25% Work Arrangement This is a fully onsite position requiring the successful candidate to work from the Toronto office at 222 Jarvis St. 5 days per week. Ideal Candidate Profile The ideal candidate combines strong Python and data science expertise with practical experience in Azure Databricks MLflow and modern LLM frameworks. The candidate should be comfortable working hands-on with datasets inference pipelines model evaluation experimentation and analytical solutions while also communicating findings effectively to government stakeholders and business users. Mandatory technology stack: Azure Databricks MLflow Python pandas/scikit-learn/PyTorch LLM framework exposure (DSPy LangChain Hugging Face or equivalent).
Required Education:
Education RequirementsBachelors degree or higher in:Computer ScienceData ScienceArtificial IntelligenceMachine LearningStatisticsMathematicsor a closely related fieldAdvanced degree (Masters/PhD) in Data Science AI/ML Statistics or a related discipline would be an advantage.