Senior AI Engineer (Regression Evaluation & Model Quality Systems)

Bengaluru - India

Monthly Salary: Not Disclosed

Posted on: 10 hours ago

Vacancies: 1 Vacancy

Job Summary

Senior AI Engineer (Regression Evaluation & Model Quality Systems)

Location: Flexible (India)

Type: Full-time

Company: Zscaler

About Zscaler For over a decade Zscaler has been redefining cloud security by delivering a globally distributed cloud-native platform that secures users applications and data across 185 countries. Protecting more than 7000 enterprises and detecting over 150 million threats daily Zscalers mission is to make secure digital transformation seamless. We thrive in a fast-paced innovative culture built on collaboration creativity and accountability. Our teams consist of some of the brightest minds in technology-driven by the challenge of building intelligent scalable and secure systems that power the modern enterprise. About the Team The AI and Data Science team leads Zscalers enterprise data intelligence strategy focusing on building AI-driven systems that enhance automation trust and decision-making. We design and implement scalable ML pipelines evaluation frameworks and generative AI tools to measure monitor and optimize model performance across the companys ecosystem. Our group bridges machine learning research data engineering and platform development creating the foundation for reliable and explainable AI across predictive analytics cybersecurity and enterprise automation. Role Overview We are seeking a Senior AI Engineer with a strong foundation in regression modeling evaluation systems and AI this role you will design and build the pipelines that measure and ensure the reliability of large-scale AI systems-including large language models (LLMs) generative AI services and predictive ML models. You will collaborate with data scientists ML engineers and DevOps teams to develop scalable automated evaluation systems that quantify model performance detect drift and ensure statistical consistency across diverse business and technical use cases.

Key Responsibilities

Design implement and automate regression-based evaluation systems for large language models and predictive ML services.

Develop statistical and ML pipelines to monitor model drift bias and performance trends across large-scale datasets.

Analyze regression outputs and experiment data to identify key quality drivers and actionable insights for model improvement.

Partner with ML and product teams to benchmark and fine-tune models for applications including generative AI anomaly detection and customer analytics.

Collaborate with DevOps and platform teams to deploy evaluation and monitoring tools on AWS or GCP infrastructure (EKS ECS SageMaker Vertex AI).

Build dashboards and reporting frameworks to visualize regression performance metrics (RMSE MAPE R stability indices etc.).

Document evaluation methodologies and standardize reproducible model validation frameworks across teams.

Stay current with advances in regression evaluation LLM testing and scalable ML performance measurement.

Qualifications

Bachelors or Masters degree in Computer Science Data Science Statistics or a related technical field.

5 years of hands-on experience in AI/ML engineering data science or model evaluation with emphasis on regression analysis and model reliability. Strong proficiency in Python (pandas NumPy scikit-learn PyTorch/TensorFlow).

Solid understanding of machine learning evaluation techniques including performance metrics A/B testing and model drift analysis.

Experience with cloud platforms (AWS or GCP) and deploying evaluation systems using EKS ECS SageMaker or Vertex AI.

Familiarity with LLM frameworks (LangChain RAG architectures vector databases) or generative AI evaluation is a strong plus.

Demonstrated ability to build automated regression testing and monitoring systems for production ML pipelines.

Preferred Skills

Experience with MLOps CI/CD for ML pipelines and monitoring platforms (e.g. MLflow Weights & Biases or Prometheus).

Strong statistical background with expertise in time-series regression panel data or causal inference.

Knowledge of data visualization and reporting tools (Plotly Streamlit Power BI).

Excellent collaboration and communication skills with cross-functional teams.

A passion for reliable explainable and measurable AI systems.

Senior AI Engineer (Regression Evaluation & Model Quality Systems) Location: Flexible (India) Type: Full-time Company: Zscaler About Zscaler For over a decade Zscaler has been redefining cloud security by delivering a globally distributed cloud-native platform that secures users applications and ...