Design and Implementation of ZhiZhuan's Low-Result Search Module with Hybrid Hard and Soft Retrieval

In any app's search scenario, relevance and traffic efficiency are core goals; ZhiZhuan's low-result module serves as a fallback to the main search, displaying "You May Like" items when the main search yields few or no results, effectively acting as a recommendation layer.

The module tolerates lower relevance compared to the main search and aims to (1) recommend similar items when inventory is insufficient for a clear user demand, and (2) present inferred product sets when the main search cannot accurately capture niche queries.

The overall architecture comprises a query understanding component, a recall layer, a ranking layer, and a business re‑ranking layer. The core logic relies on query understanding and the recall layer, which includes both text‑based ElasticSearch recall and semantic vector recall.

For hard matching, the custom "low‑result QR" module preprocesses the user query by removing stop words, normalizing, simplifying, and expanding it into multiple variants (e.g., converting "Iphone12 PM 256北京发货" to standardized, simplified, and expanded forms). These variants are then fed into a customized ElasticSearch multi_match query that uses the best_fields strategy and field‑weighting to retrieve the most relevant product documents.

Because hard matching struggles with colloquial or domain‑specific expressions in second‑hand scenarios, a soft‑matching layer based on an sBert twin‑network is introduced. A cited 2022 paper shows that while BERT underperforms BM25 on exact term matches, it excels at semantic generalization, retrieving documents missed by BM25.

The system builds a Faiss index of product vectors updated daily; at query time, both the ElasticSearch hard recall and the sBert semantic recall are executed, their results filtered by category, and finally re‑ranked before presentation.

The sBert model combines two shared BERT towers—one for the user query and one for product attributes—producing sentence embeddings that are pooled (Avg or Mean) and compared via cosine similarity. Training uses mean‑square error loss on triplet data, achieving AUC scores above 0.8 (up to 0.92 on random recall sets).

Negative sample selection is critical: easy negatives are generated randomly from query‑product attribute pairs, while hard negatives are sampled using a dynamic step size and a query‑class mapping derived from recent search logs, ensuring that the model learns to discriminate between highly similar items.

Extensive online experiments confirmed that the combined hard‑soft recall significantly improves coverage: low‑result query coverage increased by 10%, product exposure grew by nearly 30%, and order volume roughly doubled, validating the effectiveness of the hybrid approach.

References

[1] Rau D, Kamps J. How Different are Pre‑trained Transformers for Text Ranking? 2022. [2] Huang P S, He X, Gao J, et al. Learning Deep Structured Semantic Models for Web Search Using Clickthrough Data. ACM, 2013. [3] Yi X, Yang J, Hong L, et al. Sampling‑bias‑corrected Neural Modeling for Large Corpus Item Recommendations. ACM, 2019. [4] Huang J T, Sharma A, Sun S, et al. Embedding‑based Retrieval in Facebook Search. 2020. [5] 石塔西: 深度学习中不得不学习的 Graph Embedding 方法. https://zhuanlan.zhihu.com/p/165064102