How We Cut Log System Size by 70% with Smart Cleanup, Async Loading, and Build Optimizations

Introduction

In modern frontend applications, log retrieval systems are essential for debugging, but traditional implementations suffer from large bundle size, uncontrolled storage growth, and performance impact.

Core Optimization Strategies

Storage bloat – intelligent cleanup to limit local storage.

Bundle size – async module loading for on‑demand code.

Performance impact – queue and throttling mechanisms.

Optimization 1: Smart Database Cleanup

Problem: IndexedDB accumulates tens of thousands of logs, bloating storage and slowing queries.

Solution: Dual‑phase cleanup that removes logs older than a configurable retention period and trims excess entries when a threshold is exceeded, executed only during browser idle time.

/**
 * Comprehensive log cleanup (handles expiration and count limits)
 * @param retentionDays Retention days
 * @param maxLogCount Maximum log count
 */
async cleanupLogs(retentionDays?: number, maxLogCount?: number): Promise<void> {
  if (!this.db) {
    throw new Error('Database not initialized');
  }
  try {
    // First clean expired logs
    if (retentionDays && retentionDays > 0) {
      await this.clearExpiredLogs(retentionDays);
    }
    // Then clean excess logs
    if (maxLogCount && maxLogCount > 0) {
      await this.clearExcessLogs(maxLogCount);
    }
  } catch (error) {
    // Cleanup failures should not affect main flow
    console.warn('Log cleanup failed:', error);
  }
}

Additional throttling ensures cleanup runs at most once every five minutes.

Optimization 2: Asynchronous Upload Module

Problem: Upload functionality (OSS upload, file compression) adds ~189 KB to the main bundle, yet most users never trigger it.

Solution: Separate the upload logic into its own library, host it on a CDN, and load it dynamically via a script tag only when the user initiates an upload. A singleton loader guarantees a single request.

// OSS uploader URL
const OSS_UPLOADER_URL = 'https://cdn-jumper.dewu.com/sdk-linker/dw-log-upload.js';

// Load remote module once (singleton)
let moduleLoadPromise: Promise<any> | null = null;
export const loadRemoteModule = async (): Promise<any> => {
  if (!moduleLoadPromise) {
    moduleLoadPromise = (async () => {
      try {
        await loadScript(OSS_UPLOADER_URL);
        return (window as any).DWLogUpload;
      } catch (error) {
        moduleLoadPromise = null;
        throw error;
      }
    })();
  }
  return moduleLoadPromise;
};

export const uploadToOss = async (file: File, curEnv?: string, appId?: string): Promise<string> => {
  try {
    if (!ossUploader) {
      const module = await loadRemoteModule();
      ossUploader = module.uploadToOss;
    }
    const result = await ossUploader(file, curEnv, appId);
    return result;
  } catch (error) {
    console.info('Failed to upload file to OSS:', error);
    return '';
  }
};

Optimization 3: Dynamic JSZip Import

Problem: Including JSZip in the main bundle inflates size.

Solution: Remove JSZip from the core bundle and load it lazily inside the upload module, preferring a global window.JSZip if already present.

// Get JSZip instance
export const getJSZip = async (): Promise<JSZip | null> => {
  try {
    if (!JSZipCreator) {
      const module = await loadRemoteModule();
      JSZipCreator = module.JSZipCreator;
    }
    const zipInstance = new (window as any).JSZip();
    return zipInstance;
  } catch (error) {
    console.info('Failed to create JSZip instance:', error);
    return null;
  }
};

export const JSZipCreator = async () => {
  if ((window as any).JSZip) {
    return (window as any).JSZip;
  }
  return JSZip;
};

Optimization 4: Log Queue and Throttling

Problem: Rapid error loops cause a burst of IndexedDB writes, blocking the main thread.

Solution: Buffer log writes in a queue, limit processing to 50 logs per second (error level logs are exempt), batch writes using requestIdleCallback, and execute during idle periods.

export class LogQueue {
  private readonly MAX_LOGS_PER_SECOND = 50;
  private logCount = 0;
  private lastResetTime = 0;

  /** Check rate‑limit logic */
  private checkRateLimit(entry: LogEntry): boolean {
    // Error logs are always accepted
    if (entry.level === 'error') {
      return true;
    }
    const now = Date.now();
    if (now - this.lastResetTime > 1000) {
      this.logCount = 0;
      this.lastResetTime = now;
    }
    if (this.logCount >= this.MAX_LOGS_PER_SECOND) {
      return false;
    }
    this.logCount++;
    return true;
  }
}

export function executeWhenIdle(callback: () => void, timeout: number = 2000): void {
  if (typeof window !== 'undefined' && 'requestIdleCallback' in window) {
    window.requestIdleCallback(() => {
      callback();
    }, { timeout });
  } else {
    setTimeout(callback, 50);
  }
}

Build‑time Challenges and Solutions

During migration we faced three major bundling issues:

Dynamic import() failing in UMD builds – solved with inlineDynamicImports: true in Rollup.

Missing process object in the browser – injected a shim via @rollup/plugin-inject.

ESM/CJS compatibility for third‑party libraries – handled with mixed import logic and Rollup plugins.

// rollup.config.js (excerpt)
export default {
  input: 'src/index.ts',
  output: [
    { file: 'dist/umd/dw-log.js', format: 'umd', name: 'DwLog', inlineDynamicImports: true },
    { file: 'dist/cjs/index.js', format: 'cjs', inlineDynamicImports: true }
  ],
  plugins: [
    typescript(),
    resolve({ browser: true }),
    commonjs(),
    inject({ process: path.join(__dirname, 'process-shim.js') })
  ]
};

Performance Test Results

The optimizations reduced the bundle from 189 KB to under 30 KB and eliminated noticeable UI lag during log operations.

Conclusion

By addressing storage, bundle size, and runtime performance, the logging system became lightweight and responsive, providing a reusable pattern for any frontend project that demands high performance and stability.