How Large Language Models Automate Order Cancellation Responsibility at HuoLala

Background

HuoLala, a freight transaction platform, needs to standardize driver behavior and improve service quality by assigning responsibility for order cancellations, enabling appropriate compensation or education measures.

Example: a user requested a tailboard, the driver accepted the order, then added a 100‑yuan surcharge citing the tailboard, the user refused, leading to cancellation.

Cancellation responsibility is divided between driver‑responsible and user‑responsible cases, with dozens of scenarios and hundreds of detailed rules involving driver, user, and cargo dimensions. The rules evolve regularly, making the problem complex, so a human‑machine collaborative approach is used.

All cancellations first pass through a machine that predicts relevant rules; high‑confidence rules are output directly, while low‑confidence ones undergo manual review.

System Architecture

The architecture consists of three layers:

Feature layer: gathers multidimensional information via numerous business interfaces.

Model layer: applies rule‑based strategies, natural language processing, image and speech recognition to predict relevant responsibility scenarios and rules.

Business layer: filters and combines logic to produce the final responsible party.

Large Model Solution

Recent advances in large models provide strong capabilities in complex context understanding, logical reasoning, and numeric computation, which are essential for solving cancellation responsibility challenges.

Case: a user asks the driver to help with loading, and both parties haggle over the moving fee during a call, failing to reach an agreement and causing cancellation. The platform must evaluate whether the proposed fee complies with its standards, which depend on item type, weight, distance, and floor level.

Traditional models struggle with such problems, prompting exploration of large‑model approaches.

Processing Flow

Convert multidimensional features into unified textual representations, including call recordings, photos, etc.

Retrieve the most relevant responsibility rules from a library of hundreds of rules using a retrieval model that selects the top‑N matches.

Enhance accuracy by building a case library and using another retrieval model to find similar historical examples.

Feed the textual features, retrieved rules, and similar cases into the large model for inference, producing the final responsibility decision.

Multidimensional Feature Integration

Features include:

Multimodal data: voice calls (rich content), photos of loading/unloading, and user remarks or IM messages.

Vehicle attributes: size, model, load capacity, seat count, presence of high‑bars or tailboards.

Order details: driver’s pickup location, loading/unloading points, distances, fees, navigation routes, trajectory, and user‑specified requirements such as moving assistance or accompanying passengers.

Retrieving Relevant Rules

Because the rule set is extensive, a high‑efficiency retrieval model is designed to quickly fetch the top‑N most relevant rules for a given text input, avoiding the need to include all rules in the prompt.

The retrieval model uses text matching and contrastive learning: both the input text and rule definitions are encoded by the same encoder into embeddings. Positive pairs (matching rule and text) are pulled together, while negative pairs are pushed apart, enabling fast nearest‑neighbor search at inference.

Retrieval Sample Library

A rich case library is built following a Retrieval‑Augmented Generation (RAG) approach. Each case records the textual information, a summary of the cancellation reason, rule match analysis, and rule match results.

When a new cancellation text arrives, a dedicated retrieval model finds similar cases. Only the summary, analysis, and results are kept for prompting, reducing length while preserving essential guidance. Incorporating RAG improves model precision, and continuously adding bad cases to the library helps prevent repeat errors.

Final Output

The task requirements, multidimensional information, rule definitions, and reference cases are combined into a single prompt for the large model. The output includes a cancellation reason summary and rule‑match analysis as intermediate steps, using chain‑of‑thought prompting to mitigate hallucinations and provide explainable results for business stakeholders.

Future Work

Open challenges include improving ASR quality for call transcripts, narrowing the performance gap between the model and human reviewers, and exploring better ways to integrate visual and map data beyond textual conversion.