How to Build a Lightweight Frontend Monitoring SDK with ELK Integration

1. Requirement Background: Why Frontend Needs Monitoring

Front‑end engineers often lack visibility into daily page views, feature usage peaks, and cannot argue for requirements or troubleshoot issues without data; monitoring provides the first step from a pure front‑end developer to a full‑stack engineer.

2. Requirement Analysis: What to Monitor

We need to know user activity (PV, UV, visit time, duration), snapshots for debugging (request details, page errors), and real performance data (page and resource load times).

3. Overview Design: How to Design Monitoring

The system consists of three main blocks: data collection, log reporting, and log querying.

Data collection: SDK in the browser gathers metrics.

Log reporting: HTTP requests send logs to the server.

Log querying: UI for searching logs, typically using ELK.

ELK (Elasticsearch, Logstash, Kibana) provides storage, processing, and visualization.

4. Detailed Design: Implementing the Monitoring SDK

4.1 Defining the Usage

The SDK should be usable with a single script tag:

<script src="xxxx/{version}/jssdk.js?token=123" async></script>

4.2 SDK Structure

4.3 SDK Initialization

The entry point is a self‑executing main function that extracts query parameters and calls initSDK:

(function(){
  const params = getScriptQuery();
  initSDK(params);
})();

initSDK

merges default config, script query, and global config, then starts each monitoring module:

function initSDK(opt) {
  const config = assign({
    sendPV: true,
    sendApi: true,
    sendResource: true,
    sendError: true,
    sendPerf: true,
  }, opt, window.$watchDogConfig);
  window.$watchDogConfig = config;
  config.sendPV && watchPV(config);
  config.sendApi && watchApi(config);
  config.sendResource && watchResource(config);
  config.sendError && watchError(config);
  config.sendPerf && watchPerf();
  watchCustom();
}

4.4 Monitoring Items Implementation

4.4.1 API Monitoring

Hijack XMLHttpRequest prototype to capture method, URL, timing, and status, then report via sender.report('api', ...). Fetch API can be similarly wrapped (implementation omitted).

export function watchApi(config) {
  function hijackXHR() {
    const proto = window.XMLHttpRequest.prototype;
    const originalOpen = proto.open;
    const originalSend = proto.send;
    proto.open = function(method, url) {
      this._ctx = { method, url: url || '', start: getNow() };
      return originalOpen.apply(this, arguments);
    };
    proto.send = function(body) {
      const that = this;
      const ctx = that._ctx;
      function handler() {
        if (ctx && that.readyState === 4) {
          const url = that.responseURL || ctx.url;
          if (url.indexOf(SERVER_HOST) >= 0) return;
          sender.report('api', [{
            url,
            httpMethod: ctx.method,
            httpCode: that.status,
            time: getNow() - ctx.start,
          }]);
        }
      }
      const originalFn = that.onreadystatechange;
      if (originalFn && isFunction(originalFn)) {
        that.onreadystatechange = function() {
          handler.apply(this, arguments);
          originalFn.apply(this, arguments);
        };
      } else {
        that.onreadystatechange = handler;
      }
      return originalSend.apply(this, arguments);
    };
  }
  window.XMLHttpRequest && hijackXHR();
}

4.4.2 JS Error Monitoring

Listen to window.onerror, preserve any existing handler, and report error details.

export function watchError(config) {
  const originalOnError = window.onerror;
  function errorHandler(message, source, lineno, colno, error) {
    if (originalOnError) {
      try { originalOnError.call(window, message, source, lineno, colno, error); } catch(e) {}
    }
    if (error != null) {
      sender.report('error', [{
        message,
        file: source || '',
        line: '' + (lineno || ''),
        col: '' + (colno || ''),
        stack: error.stack || '',
      }]);
    }
  }
  window.onerror = function(message, source, lineno, colno, error) {
    errorHandler(message, source, lineno, colno, error);
  };
}

4.4.3 Resource Loading Monitoring

Periodically collect performance.getEntriesByType('resource'), filter static resources, and report URL, status, and duration.

export function watchResource(config) {
  function reportAssets() {
    if (isFunction(performance.getEntriesByType)) {
      const entries = performance.getEntriesByType('resource');
      const resourceEntries = arrayFilter(entries, entry =>
        ['fetch','xmlhttprequest','beacon'].indexOf(entry.initiatorType) === -1);
      if (resourceEntries.length) {
        sender.report('resource', arrayMap(resourceEntries, entry => ({
          url: entry.name,
          httpCode: 200,
          time: Math.round(entry.duration),
        })));
      }
      if (isFunction(performance.clearResourceTimings)) performance.clearResourceTimings();
      setTimeout(reportAssets, 2000);
    }
  }
  if (document.readyState === 'complete') reportAssets();
  else addEventListener(window, 'load', reportAssets);
}

4.4.4 Page Performance Monitoring

Collect performance.timing data once the page is ready and report it.

4.4.5 Page PV Monitoring

Report page view on load and on SPA route changes (hashchange, pushState/replaceState).

export function watchPV(config) {
  let lastVisit = '';
  function onLoad() { sender.reportPV(); lastVisit = location.href; }
  function onHashChange() {
    sender.reportPV({ spa: config.spa, from: lastVisit });
    lastVisit = location.href;
  }
  addEventListener(window, 'hashchange', onHashChange);
  addEventListener(window, 'load', onLoad);
  addEventListener(window, 'beforeunload', function(){
    removeEventListener(window, 'hashchange', onHashChange);
    removeEventListener(window, 'load', onLoad);
  });
}

4.4.6 Custom Reporting

Expose a global $watchDogEvents array that the page can push custom events into; the SDK replaces it with an object that forwards pushes to the server.

// Define global array before SDK loads
window.$watchDogEvents = window.$watchDogEvents || [];
window.$watchDogEvents.push(['eventName','field1','field2']);

class CustomEventTrigger {
  push(args) {
    if (Array.isArray(args) && args[0]) {
      sender.report('custom', [{
        ext1: args[0],
        ext2: args[1] || '',
        ext3: args[2] || '',
        ext4: args[3] || '',
        ext5: args[4] || '',
      }]);
    }
  }
}
export function watchCustom() {
  const originalLogs = window.$watchDogEvents || [];
  const trigger = new CustomEventTrigger();
  window.$watchDogEvents = trigger;
  setTimeout(() => {
    for (let i = 0; i < originalLogs.length; i++) trigger.push(originalLogs[i]);
  }, 0);
}

4.5 Log Reporting Design

4.5.1 Reporting Requests

Logs are sent via POST; to avoid excessive connections, a queue batches logs every 2 seconds.

let queue = [];
let timer = null;
function sendByXhr(body) {
  const xhr = new XMLHttpRequest();
  xhr.open('POST', `${SERVER_HOST}/api/collect`);
  xhr.onreadystatechange = function() {
    if (xhr.readyState === 4 && +xhr.status !== 200) {
      const retry = body._re || 0;
      if (retry < MAX_RETRY_OF_REPORT_LOG) {
        body._re = retry + 1;
        sendLog(body);
      }
    }
  };
  xhr.withCredentials = true;
  xhr.setRequestHeader('Content-Type', 'application/json;charset=UTF-8');
  xhr.send(JSON.stringify(body));
}
function batchFlushQueue() {
  clearTimeout(timer);
  timer = null;
  const merged = mergeLogs(queue);
  sendByXhr(merged);
  queue = [];
}
export function sendLog(log) {
  queue.push(log);
  if (!timer) timer = setTimeout(batchFlushQueue, 2000);
}

4.5.2 Page Unload Handling

Because POST may be blocked on unload, the SDK prefers navigator.sendBeacon with a fallback to XHR.

function sendByBeacon(body) {
  try {
    if (navigator.sendBeacon && navigator.sendBeacon(`${SERVER_HOST}/api/collect`, JSON.stringify(body))) {
      return;
    }
  } catch(e) {}
  sendByXhr(body);
}

Additional listeners such as visibilitychange and pagehide are recommended for reliability.

5. Summary and Outlook

The SDK, built under 10 KB (gzip < 4 KB), provides comprehensive front‑end monitoring. Logs stored in Elasticsearch can be visualized with Kibana or Grafana. Future work includes building a unified log‑analysis tool for user‑level tracing and extending monitoring to mini‑program ecosystems.