How Keyword-Based Scoring Boosts Sentence Similarity for Chatbots

The 9th Web Science International Conference was held in New York from June 26‑28, where Suning’s research paper "A Keyword‑Based Method for Measuring Sentence Similarity" was accepted and presented.

Web Science, an ACM‑supported conference, gathers leading scholars from Stanford, Purdue, Cambridge and industry researchers from Google, Yahoo, Facebook, covering topics such as information science, artificial intelligence, data science, software engineering, and digital humanities. Accepted papers are published in the Journal of Web Science.

The paper, authored by Suning’s Silicon Valley research team, introduces a new keyword‑based approach for measuring sentence similarity, addressing limitations of traditional word‑overlap methods that can be noisy when shared words are unimportant.

Proposed Method

The method considers three factors: keywords, part‑of‑speech, and word position. Keyword extraction helps the chatbot identify service‑relevant terms; part‑of‑speech weighting allows recognition of unseen words; positional scoring improves accuracy for long, complex sentences. A ranking score for each word is computed, and the Jaccard similarity coefficient is applied to these scores, yielding more efficient and accurate similarity calculations suitable for industrial products.

Experimental Results

Experiments built on Suning’s existing chatbot after‑sales service module show that the keyword‑based method not only improves similarity accuracy but also supports diverse sentence expressions. The approach increased relevance judgment accuracy by up to 30%.

Impact on Suning’s AI Platform

Since late 2016, Suning has leveraged AI to develop its "LiaoShang" chatbot platform, enabling rich human‑machine dialogues across e‑commerce, real‑estate, cultural‑creative, finance, and marketing scenarios. The keyword‑based similarity method enhances the platform’s ability to understand and respond to user queries, fundamentally transforming user experience and supporting a broader ecosystem of services.