Beyond 'Should Be Fine': 360's TEP Platform Automates Model Evaluation

The Problem: Relying on "Should Be Fine" Before Model Launch

Teams developing large language models often decide whether to deploy a new version by manually testing a few examples. If the outputs look acceptable, they say "should be fine." However, LLM outputs are highly sensitive to phrasing; a model may excel on a handful of cherry-picked prompts but fail on others. Moreover, evaluation scripts scatter across individuals, metrics definitions diverge, and stakeholders disagree on results. As model count and iteration speed increase, this ad-hoc sampling becomes unreliable.

The article argues that evaluation should not be a last-minute cram session but a continuous infrastructure embedded in the R&D process. 360 AI Platform's TEP (Test Evaluation Platform) automatic evaluation is built for this purpose.

Core Workflow: System Executes, Humans Define

TEP's workflow is straightforward: prepare an evaluation dataset, create an evaluation task, the system invokes the specified evaluation service, batches model outputs, analyzes them uniformly against evaluation rules, and produces a report. Humans only perform two actions: define the evaluation dataset and define the evaluation rules; the system handles the rest.

Important scope limitation: currently TEP only supports text-generation models ; image and multimodal models are not yet supported. This narrowing standardizes the process and allows text evaluation methods to be reused.

Task List: Centralized Visibility and Collaboration

The "Model Evaluation — Automatic Evaluation" entry point shows a task list. Creating a task starts a new evaluation run. The list displays task name, run status, evaluation service, evaluation type, creator, creation time — all in one view.

This list solves common collaboration friction: who created the task, when it ran, which service was used, what stage it reached, where results are. Evaluation shifts from "whoever ran the script knows" to an engineering task the whole team can query and manage .

Two-Dimensional Evaluation: Effect and Performance

"Good output" and "fast service" are distinct concerns. TEP splits automatic evaluation into two categories:

Effect Evaluation — measures whether model answers are correct and high-quality; suitable for comparing different models, prompts, or training versions.

Performance Evaluation — measures service stability and latency under real invocation; suitable for observing deployed service behavior.

Both task types run in parallel; selection decisions consider both data sets.

Evaluators: Built-In for Speed, Custom for Business Alignment

Evaluators come in two flavors:

Built-in evaluators — ready-to-use, covering standard capabilities; pick them for a quick evaluation round.

Custom evaluators — open the criteria to business and algorithm teams so each project can define "what good looks like" for its specific goals.

Additionally, TEP provides leaderboards divided into external leaderboards and platform leaderboards . Their value is horizontal comparability: all models and versions are measured under the same yardstick, eliminating vendor self-promotion.

Evaluation Feeds the Next Iteration

A completed evaluation report is not just a sign-off artifact. It helps pinpoint root causes: if effect dropped, is it the model itself or a prompt issue? If effect improved, did performance degrade?

Tasks are reproducible, reports are reviewable, and criteria are unified. This lets the team continuously compare around the same data. Training, deployment, and evaluation form a closed loop — every iteration references the previous round's data, so the next version's direction is evidence-driven, not guesswork.