Technical Lead NLP
Share
Job Location:
Portland, TX - USA
Monthly Salary:
Not Disclosed
Posted on:
6 hours ago
Vacancies:
1 Vacancy
Job Summary
Job Role: Senior Technical Lead - NLP
Location: Remote (Support in PST hours)
Experience level: 10 years
Must have skills: SDET skills Azure devops python pytest test automation and Framework AI ML practices RAGAS LLMs
Brief JD:
Practical knowledge of Databricks Azure Cloud services land distributed data validation.
Proficiency with Azure DevOps pipelines YAML templates agent pools CICD workflows.
Experience in implementing AIML practices in testing e.g. test generation anomaly detection log analysis to improve test efficiency and coverage.
Familiarity with Cucumber (BDD) and test reporting frameworks (e.g. Allure).
Strong understanding of integration testing across Databricks streaming jobs applications and microservices.
Experience with performance testing tools (JMeter LoadRunner or equivalent).
Experience with Pester framework for validating PowerShell scripts and infrastructure automation.
Solid understanding of Agile/Scrum methodology and handson usage of JIRA.
Excellent problem-solving debugging and communication skills with the ability to advise development teams on testing best practices
Must have skills: SDET skills Azure devops python pytest test automation and Framework AI ML practices RAGAS LLMs
Brief JD:
- Experience in designing LLM/RAG test automation solutions
- Experience in testing for bias drift and fairness.
- Familiarity with performance metrics (precision recall F1 ROC-AUC).
- Knowledge of MLflow MLOps framework
- Knowledge of tools for synthetic data generation and boundary testing
- Knowledge of Azure DevOps for CI/CD pipeline development
- Experience in RAG/pipeline evaluation frameworks -
- pytest: Test automation framework
- DeepEval or TruLens: LLM test assertions
- RAGAS: RAG-specific metrics
- Eleuther: LLM evaluation harness
- Garak or Promptfoo: LLM red-teaming
- Evidently: Drift/performance monitoring
- Exposure to explainability frameworks (SHAP LIME Captum)
Practical knowledge of Databricks Azure Cloud services land distributed data validation.
Proficiency with Azure DevOps pipelines YAML templates agent pools CICD workflows.
Experience in implementing AIML practices in testing e.g. test generation anomaly detection log analysis to improve test efficiency and coverage.
Familiarity with Cucumber (BDD) and test reporting frameworks (e.g. Allure).
Strong understanding of integration testing across Databricks streaming jobs applications and microservices.
Experience with performance testing tools (JMeter LoadRunner or equivalent).
Experience with Pester framework for validating PowerShell scripts and infrastructure automation.
Solid understanding of Agile/Scrum methodology and handson usage of JIRA.
Excellent problem-solving debugging and communication skills with the ability to advise development teams on testing best practices
Key Skills
- Administrative Skills
- Facilities Management
- Biotechnology
- Creative Production
- Design And Estimation
- Architecture
