Implement Full-Stack Interface Monitoring Across Web, Mini‑Programs, and Flutter

Why Do Interface Analysis

Identify which applications or pages use specific backend APIs for governance.

Notify quickly when an interface errors.

Provide metric, log, and error analysis.

Why Collect Data on the Frontend

Capture timeouts, network errors that the server cannot see.

Collect detailed information such as hash routing.

Front‑end perception of errors makes troubleshooting easier.

How to Collect Data

Web

Hook XMLHttpRequest and fetch. Example using fetch:

const originalFetch = window.fetch;
window.fetch = function (input, init) {
  // get request info
  return originalFetch.apply(window, [input, init]).then(res => {
    // get response info
    return res;
  });
};

Mini‑Program

WeChat: https://developers.weixin.qq.com/miniprogram/dev/api/network/request/wx.request.html

Alipay: https://opendocs.alipay.com/mini/api/owycmh

ByteDance: https://microapp.bytedance.com/docs/zh-CN/mini-app/develop/api/network/http/tt-request

DingTalk: https://open.dingtalk.com/document/orgapp/send-network-requests

QQ: https://q.qq.com/wiki/develop/miniprogram/API/network/network_request.html

Baidu: https://smartprogram.baidu.com/docs/develop/api/net/request/=

Hook the request’s complete callback:

const originRequest = global[method];
Object.defineProperty(global, method, {
  writable: true,
  enumerable: true,
  configurable: true,
  value(...args) {
    const options = args[0];
    const originComplete = options.complete;
    options.complete = function (res) {
      // get response info
      originComplete && originComplete(res);
    };
  }
});

Flutter

Override HttpOverrides with a custom implementation:

HttpOverrides? origin = HttpOverrides.current;
HttpOverrides.global = CustomHttpOverrides(origin: origin);

class CustomHttpOverrides extends HttpOverrides {
  final HttpOverrides? origin;
  CustomHttpOverrides({this.origin});
  @override
  HttpClient createHttpClient(SecurityContext? context) {
    if (origin != null) {
      return origin!.createHttpClient(context);
    }
    HttpOverrides.global = null;
    final httpClient = CustomHttpClient(HttpClient(context: context));
    HttpOverrides.global = this;
    return httpClient;
  }
}

Data Processing

Metric Handling

Metrics are stored using Node.js, Redis, InfluxDB, and MySQL for cost reasons.

Store in Redis

const date = new Date();
const [yy, mm, dd] = [date.getFullYear(), date.getMonth(), date.getDate()];
await redis().incrby(`${url}.${yy}${mm}${dd}`, 1);
await redis().sadd(`full-urls.${yy}${mm}${dd}`, value);

Scheduled Jobs

Periodically write aggregated data to MySQL.

// batch process URLs with SSCAN
const [, members] = await redis().sscan(skey, (page - 1) * 300, 'COUNT', 300);
// get PV for each URL
const pv = await redis().get(pvKey) || 0;
// write PV to MySQL

Trend Statistics

Cache per‑minute metrics in memory, flush to InfluxDB each minute, then query for front‑end charts.

const cache = {
  [`${minute}`]: {
    [`${url}`]: {
      pv: 0,
      rt: 0,
      // ... other metrics
    }
  }
};

Interface Logs

Logs are written to an Elasticsearch cluster. Node.js consumes Kafka messages, writes to ES, and controls concurrency.

Concurrency Control

export class ConcurrencyService {
  private queue: Function[] = [];
  public currentConcurrency = 0;
  public maxConcurrency = 2;
  public exec(fn: any) {
    this.queue.push(fn);
    this.next();
  }
  private next() {
    if (this.currentConcurrency < this.maxConcurrency && this.queue.length > 0) {
      const fn = this.queue.shift();
      this.currentConcurrency++;
      fn()
        .catch(error => console.error(error))
        .finally(() => {
          this.currentConcurrency--;
          this.next();
        });
    }
  }
}

Automatic Flow Control

if (this.queue.length <= 3 && this.paused) {
  this.paused = false;
  console.warn(`Load recovered, resume consumption`);
  this.kafka.resumeTopics(topics);
}
if (this.queue.length > 6 && !this.paused) {
  this.paused = true;
  console.warn(`Load high, pause consumption`);
  this.kafka.pauseTopics(topics);
}

Full‑Link Tracing

Backend services expose a trace‑id (e.g., via ARMS). The front‑end reports this ID, allowing correlation of requests with backend logs and visualizing the entire call chain.

Conclusion

The monitoring platform’s interface analysis module is complex but essential, handling massive data volumes and CPU‑intensive metric calculations, and it continuously evolves to address real‑world pain points.