How to Build a Versatile Canvas Library with Edge Detection, Video Capture, and Particle Effects

Introducing a Swiss‑Army Knife Canvas Library

Inspired by the limitless possibilities of the HTML5 Canvas element, I created a comprehensive library that bundles visual effects, performance optimizations, and useful utilities into a single package.

What You’ll Learn

Implementation ideas : Core techniques for cool Canvas effects such as high‑performance video screenshots, image edge detection, and pixel‑based particle animations.

Performance optimization : Using Web Worker and modern browser APIs to offload intensive calculations and avoid UI blocking.

Abstract encapsulation : How to structure the code as a reusable library.

Ready‑to‑use tools : A powerful Canvas utility library you can drop into any project.

Edge Detection with Sobel Algorithm

Image edges are the “skeleton” of visual information. To detect them we first convert the image to grayscale, then apply Sobel convolution kernels to compute horizontal ( gx) and vertical ( gy) gradients, finally calculate the gradient magnitude √(gx²+gy²) and threshold it.

type ImageData = {
  width: 100,
  height: 100,
  data: Uint8ClampedArray // 100×100×4
}

/**
 * Convert ImageData to a grayscale Uint8Array
 * @param imageData Image data
 * @returns 0~255 Uint8Array
 */
export function getGrayscaleArray(imageData: ImageData): Uint8Array {
  const grayData = new Uint8Array(imageData.width * imageData.height);
  for (let i = 0; i < imageData.data.length; i += 4) {
    const gray = Math.round(
      0.299 * imageData.data[i] +
      0.587 * imageData.data[i + 1] +
      0.114 * imageData.data[i + 2]
    );
    grayData[i / 4] = gray;
  }
  return grayData;
}

/** Sobel edge detection */
function sobelEdgeDetection(grayData: Uint8Array, width: number, height: number, threshold: number): ImageData {
  const edgeData = new ImageData(width, height);
  const sobelXKernel = [-1,0,1,-2,0,2,-1,0,1];
  const sobelYKernel = [-1,-2,-1,0,0,0,1,2,1];
  for (let y = 1; y < height - 1; y++) {
    for (let x = 1; x < width - 1; x++) {
      let gx = 0, gy = 0;
      for (let ky = -1; ky <= 1; ky++) {
        for (let kx = -1; kx <= 1; kx++) {
          const pixel = grayData[(y + ky) * width + (x + kx)];
          const idx = (ky + 1) * 3 + (kx + 1);
          gx += pixel * sobelXKernel[idx];
          gy += pixel * sobelYKernel[idx];
        }
      }
      const gradient = Math.sqrt(gx * gx + gy * gy);
      const edgeStrength = gradient > threshold ? 255 : 0;
      const i = (y * width + x) * 4;
      edgeData.data[i] = edgeStrength;
      edgeData.data[i + 1] = edgeStrength;
      edgeData.data[i + 2] = edgeStrength;
      edgeData.data[i + 3] = 255;
    }
  }
  return edgeData;
}

If the horizontal change is large, gx becomes large; if the vertical change is large, gy becomes large.

Video Frame Capture without Freezing the UI

Traditional DOM‑based capture creates a video element, seeks to the target time, draws it onto a Canvas, and calls toDataURL. All steps run on the main thread and can block rendering when many frames are needed.

The new solution combines Web Worker with the ImageCapture API to move heavy work off the UI thread.

async function genWorkerData(video: HTMLVideoElement): Promise<CaptureVideoFrameData> {
  const stream = video.captureStream();
  const track = stream.getVideoTracks()[0];
  const imageCapture = new ImageCapture(track);
  const imageBitmap = await imageCapture.grabFrame();
  const timestamp = video.currentTime;
  return { imageBitmap, timestamp, mimeType: opts.mimeType, quality: opts.quality };
}

The main thread sends the ImageBitmap (a transferable object) to the worker via postMessage. The worker creates an OffscreenCanvas, draws the bitmap, and calls convertToBlob to obtain a Blob, which is then transferred back as an ArrayBuffer.

async function getCaptureFrame(videoData: CaptureVideoFrameData) {
  const { imageBitmap, mimeType, quality } = videoData;
  const canvas = new OffscreenCanvas(imageBitmap.width, imageBitmap.height);
  const ctx = canvas.getContext('2d')!;
  ctx.drawImage(imageBitmap, 0, 0);
  return new Promise<ArrayBuffer>(resolve => {
    canvas.convertToBlob({ type: mimeType, quality })
      .then(async blob => {
        const buffer = await blob.arrayBuffer();
        imageBitmap.close();
        resolve(buffer);
      })
      .catch(reject);
  });
}

This approach offers near‑zero copy transfer, dramatically reducing main‑thread workload and outperforming ffmpeg.wasm in both speed and bundle size.

Particle‑Based “Fade‑to‑Dust” Effect

The effect simulates an image disintegrating into moving particles. Two canvases are used: a visible background canvas and an off‑screen canvas that holds the original image data.

export async function imgToFade(bgCanvas: HTMLCanvasElement, opts: ImgToFadeOpts) {
  const { width, height, imgWidth, imgHeight, img } = await checkAndInit(opts);
  const bgCtx = bgCanvas.getContext('2d')!;
  bgCanvas.width = width;
  bgCanvas.height = height;
  const { cvs: imgCvs, ctx: imgCtx } = createCvs(imgWidth, imgHeight);
  imgCtx.drawImage(img, 0, 0, imgWidth, imgHeight);
  // ... animation loop follows
}

All pixel indices are stored in an array; each animation frame randomly selects a pixel, creates a colored Ball particle at that location, clears the pixel from the off‑screen canvas, and moves the particle outward. An optional “extra delete” step accelerates disappearance.

function createAndDelParticle(size: number) {
  for (let i = 0; i < size; i++) {
    const [x, y, index] = getXY();
    const [R, G, B, A] = getPixel(x, y, imgData);
    const color = `rgba(${R}, ${G}, ${B}, ${A})`;
    const point = new Ball({ x: x + centerX, y: y + centerY, color });
    destroyBalls.push(point);
    clearPixel(x, y, index);
    // extra deletions without particles
    for (let j = 0; j < extraDelCount; j++) {
      const [ex, ey, eIdx] = getXY();
      clearPixel(ex, ey, eIdx);
    }
  }
}

Feature‑Rich Online Drawing Board (NoteBoard)

The library also provides a full‑featured drawing board with multiple modes (brush, erase, rectangle, circle, arrow), canvas operations (drag, zoom, right‑click drag), history (undo/redo), layer management, image import/export, high‑DPI support, and extensible hooks.

abstract class NoteBoardBase {
  el: HTMLElement;
  canvas = document.createElement('canvas'); // pen layer
  ctx = this.canvas.getContext('2d')!;
  imgCanvas = document.createElement('canvas'); // background layer
  imgCtx = this.imgCanvas.getContext('2d')!;
  constructor(opts: NoteBoardOptions) {
    this.el = opts.el;
    this.canvas.style.zIndex = '20';
    this.imgCanvas.style.zIndex = '10';
    this.el.appendChild(this.imgCanvas);
    this.el.appendChild(this.canvas);
  }
  async setTransform(callback) {
    const transformOrigin = `${this.mousePoint.x}px ${this.mousePoint.y}px`;
    const transform = `scale(${this.scale}, ${this.scale}) translate(${this.translateX}px, ${this.translateY}px)`;
    this.canvasList.forEach(item => {
      item.canvas.style.transformOrigin = transformOrigin;
      item.canvas.style.transform = transform;
    });
    callback?.({ transform, transformOrigin });
  }
}

Undo/redo can be implemented either by snapshotting the pen layer as a Base64 image (simple but memory‑heavy) or by recording user commands (paths, shapes) and replaying them (memory‑efficient and extensible). The command‑based approach is preferred for long‑term projects.

Using the Library

Install via npm:

# pnpm
pnpm add @jl-org/cvs

# npm
npm i @jl-org/cvs

Clone the repository for demos:

# Clone
git clone https://github.com/beixiyo/jl-cvs.git

# Install
pnpm install

# Build core package
pnpm build

# Run test page
pnpm test