GrowLoop: Turning Subjective Dialogue Quality into a Rational Benchmark

Why Open‑Domain Dialogue Evaluation Is Hard

Assessing how human‑like an AI conversation feels lacks a clear, quantifiable standard. Human annotators disagree (overall agreement 51.1%), because judgments about empathy, appropriateness, and distance are tied to personal experience and cultural background. Moreover, tacit knowledge makes it impossible to fully write down criteria, and standards drift as models improve and user expectations rise.

Existing Solutions

Three main approaches have been tried:

Hand‑written rubrics : Experts define dimensions and weights, but for "human‑like" dialogue there is no expert consensus and the static rules quickly become outdated.

Reward‑model training : Collect pairwise preference data and train a scoring model. In conversational settings the correlation with human judgments is negative (e.g., RM‑R1 = –0.50, Skywork‑Reward‑V2 = –0.20) because the models favour detail and logical completeness, not the brevity and emotional alignment humans expect.

Automatic question difficulty increase : Generate harder test items, but this does not solve the core problem that the evaluation metric itself is ill‑defined.

All three miss the fundamental issue: the standard itself is the problem.

GrowLoop’s Core Idea

GrowLoop treats the scoring rubric as a latent object that can be learned by the LLM. A small set of human‑annotated seed examples provides initial signals. The model then reflects on why its own scores differ from human judgments, extracts the hidden rules, and updates the rubric. The updated rubric is used to generate new test questions, which in turn expose blind spots in the rubric, creating a virtuous cycle of co‑evolution.

First Trick: Admit That Consensus May Not Exist

Questions are split into two zones:

Consensus zone : All annotators agree; the model must match the human answer.

Divergence zone : No agreement; the model only needs to produce a judgment that lies within the reasonable human opinion range (“reasonable” rather than “correct”).

Example: a medical advice response was marked fatal by the model (role‑overstepping) while three annotators missed the issue, showing that the model can surface judgments humans overlook.

Second Trick: Heuristic Learning via Meta‑Reflection

The process, called “Heuristic Learning”, repeats four steps:

Score : The LLM scores each question using the current rubric.

Compare : Compare LLM scores with human annotations.

Reflect : The LLM explains why its score deviated, identifying vague rubric items or missing dimensions.

Revise : Update the rubric based on the reflection and repeat.

The loop stops when safety agreement reaches 90% and quality agreement 85%.

Fatal issue: assign 0 points when detected</code><code>Fatal issue list: ...</code><code>Hallucination issue: ...

The refined rubric includes a four‑layer meta‑cognitive framework (purpose, consequence, value, rule) that forces the model to consider the real intent, short‑ and long‑term effects, safety vs. authenticity trade‑offs, and concrete rule checks.

Third Trick: Co‑Evolving Rubric and Questions

After a rubric converges, it is used to generate a large batch of new questions (e.g., 500 items). Models of four capability tiers (Claude Opus 4.7, Qwen3.5‑Plus, Qwen3‑235B, Qwen3‑80B) answer them. Sampling shows 100% consistency in the ordering “strongest > good > medium > weak”, proving the questions reliably distinguish ability levels.

Empirical Results

On 132 questions and 355 paired judgments, GrowLoop achieved 0.78 on the strictest metric, beating the second‑best method (ICAI) at 0.58.

In a cooking‑assistant scenario, GrowLoop correctly chose the concise answer (B) over the verbose one (A), learning the rule that “response length must match context pressure”.

Unexpected Findings

In divergence zones the model can provide a judgment angle that no annotator considered (e.g., detecting role‑overstepping in a medical advice example). This does not mean the model is “more correct”; rather, it reduces the cost of human reflection by surfacing overlooked concerns.

Limitations

Only one form of rubric‑question decoupling has been validated; other forms (per‑question bespoke rubrics) remain untested.

The current system is not yet integrated into reinforcement‑learning pipelines, so its impact on downstream model training is still hypothetical.

The method relies on textual inputs; extending to multimodal evaluation (audio tone, visual design) requires more capable multimodal LLMs.

Future Work and Applications

Planned steps include distilling the evolving rubric into a compact reward model, plugging it into RL training, and iterating as new failure modes appear. Beyond dialogue, the approach applies to domains where judgments are holistic and hard to formalize, such as research peer review, art assessment, and educational evaluation.

GrowLoop thus offers a general framework for turning tacit, subjective standards into a self‑improving benchmark infrastructure.