Build a WeChat Mini‑Program for Automatic Speech Evaluation with iHearing Plugin

Tencent recently released the iHearing speech‑evaluation plugin for WeChat mini‑programs, which can automatically score pronunciation, detect errors, and support users from children to adults across various assessment modes such as words, sentences, paragraphs, free speech, and dialogue.

The open‑source repository https://github.com/Tencent/iHearing provides a complete implementation of the plugin. By adding the plugin in the WeChat console (Settings → Third‑party Services → Add Plugin) and searching for "智聆口语评测", developers can include it in their mini‑program.

In app.json specify the plugin version and provider:

{
  "plugins": {
    "ihearing-eval": {
      "version": "0.0.3",
      "provider": "wx63d6102e26f0a3f8"
    }
  }
}

Import the plugin in index.js and obtain the global speech‑recognition manager:

const plugin = requirePlugin("ihearing-eval");
const manager = plugin.getRecordRecognitionManager();

Word Evaluation

The word mode evaluates a single word with detailed metrics such as phoneme accuracy, word accuracy, and fluency, and can highlight mispronounced phonemes.

Sentence Evaluation

The sentence mode assesses an entire sentence, providing overall word accuracy, sentence completeness, and fluency information. It can also generate statistics for each word in the sentence.

UI Markup

Typical WXML for displaying results includes views for the assessment item, phonetic display, progress bars, and a button to start/stop recording:

<view class="container">
  <view class="panel">
    <view class="assessment-wrap">
      <text class="assessment-item">{{assessmentItem.content}}</text>
      <text class="phonetic" wx:if="!hasResult">{{assessmentItem.phonetic}}</text>
      <view class="result-bg result" wx:if="hasResult">
        <view class="source">{{overallResult.pron_accuracy}}</view>
        <view class="result-pron">
          <view class="pron-item" wx:for="{{overallIndex}}" wx:key="key">
            <text class="pron-label">{{item.desc}}</text>
            <progress class="pron-progress" percent="{{overallResult[item.key]}}" />
            <text class="pron-val">{{overallResult[item.key]}}%</text>
          </view>
        </view>
      </view>
      <view class="bottom">
        <view class="btn-speak" catchtouchstart="streamRecord" catchtouchend="streamRecordEnd">按住跟读</view>
      </view>
    </view>
  </view>
</view>

Event Binding and Result Handling

Start and stop recording with manager.start() and manager.stop(). Bind onStop to process the result, handling empty results, extracting phoneme information, and updating the UI state. Bind onError for error logging.

manager.onStop = (res) => {
  const result = res.result;
  if (result.words.length === 0) {
    this.showRecordEmptyTip();
    return;
  }
  this.buildWordResult(result);
  this.setData({ hasResult: true });
  wx.hideLoading();
};

manager.onError = (res) => {
  console.log("out error", res);
};

Utility functions such as handlePhoneInfo, handleOverallResult, and buildWordResult transform raw scores into user‑friendly percentages and populate the phoneticArr for visual feedback.

For full implementation details, refer to the open‑source code and the plugin development documentation.