Boosting High‑Throughput Java Systems: Buffers, Caches, and Object Pools

High‑throughput systems are compared to project planning: many parallel tasks share a critical path that determines overall response time, which consists of CPU work, I/O, and external service latency.

From a user’s perspective, response time is the interval from clicking a button to seeing the result; short response times improve user experience. Techniques such as progressive data display can hide backend processing time.

Throughput (QPS/TPS) and concurrency have limits; exceeding them causes performance degradation due to context switching and memory pressure.

Buffer (缓冲)

A buffer is a dedicated memory area that smooths performance differences between producer and consumer components, similar to a funnel. Using a buffer reduces waiting time for slower components and speeds up overall processing.

BufferedWriter example

import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Image;
import javax.swing.JApplet;

public class NoBufferMovingCircle extends JApplet implements Runnable {
    Image screenImage = null;
    Thread thread;
    int x = 5;
    int move = 1;
    public void init() {
        screenImage = createImage(230,160);
    }
    public void start() {
        if (thread == null) {
            thread = new Thread(this);
            thread.start();
        }
    }
    @Override
    public void run() {
        try {
            while (true) {
                x += move;
                if ((x > 105) || (x < 5)) {
                    move *= -1;
                }
                repaint();
                Thread.sleep(10);
            }
        } catch (Exception e) { }
    }
    public void drawCircle(Graphics gc) {
        Graphics2D g = (Graphics2D) gc;
        g.setColor(Color.GREEN);
        g.fillRect(0, 0, 200, 100);
        g.setColor(Color.red);
        g.fillOval(x, 5, 90, 90);
    }
    public void paint(Graphics g) {
        g.setColor(Color.white);
        g.fillRect(0, 0, 200, 100);
        drawCircle(g);
    }
}

Adding a double‑buffer improves rendering smoothness:

public class BufferMovingCircle extends NoBufferMovingCircle {
    Graphics doubleBuffer = null; // buffer
    public void init() {
        super.init();
        doubleBuffer = screenImage.getGraphics();
    }
    public void paint(Graphics g) { // use buffer
        doubleBuffer.setColor(Color.white);
        doubleBuffer.fillRect(0, 0, 200, 100);
        drawCircle(doubleBuffer);
        g.drawImage(screenImage, 0, 0, this);
    }
}

Buffered I/O

Java provides two basic I/O families: InputStream/OutputStream and Reader/Writer . Wrapping them with buffered variants (e.g., BufferedInputStream, BufferedOutputStream, BufferedReader, BufferedWriter) dramatically improves file read/write speed.

public class StreamVSBuffer {
    public static void streamMethod() throws IOException {
        long start = System.currentTimeMillis();
        DataOutputStream dos = new DataOutputStream(new FileOutputStream("C:\\StreamVSBuffertest.txt"));
        for (int i = 0; i < 10000; i++) {
            dos.writeBytes(String.valueOf(i) + "
");
        }
        dos.close();
        DataInputStream dis = new DataInputStream(new FileInputStream("C:\\StreamVSBuffertest.txt"));
        while (dis.readLine() != null) {}
        dis.close();
        System.out.println(System.currentTimeMillis() - start);
    }
    public static void bufferMethod() throws IOException {
        long start = System.currentTimeMillis();
        DataOutputStream dos = new DataOutputStream(new BufferedOutputStream(new FileOutputStream("C:\\StreamVSBuffertest.txt")));
        for (int i = 0; i < 10000; i++) {
            dos.writeBytes(String.valueOf(i) + "
");
        }
        dos.close();
        DataInputStream dis = new DataInputStream(new BufferedInputStream(new FileInputStream("C:\\StreamVSBuffertest.txt")));
        while (dis.readLine() != null) {}
        dis.close();
        System.out.println(System.currentTimeMillis() - start);
    }
    public static void main(String[] args) throws IOException {
        streamMethod();
        bufferMethod();
    }
}

Typical output shows the buffered version is many times faster (e.g., 889 ms vs 31 ms).

Cache

A cache stores computed results in memory to avoid repeated expensive operations, reducing CPU load and response time. Popular Java cache frameworks include EHCache , OSCache, and JBossCache.

Key EHCache features: fast, simple, multi‑strategy, two‑level (memory + disk), persistence across JVM restarts, RMI‑based distribution, and Hibernate integration.

public class EHCacheDemo {
    public static void main(String[] args) {
        String fileName = "E:\\1008\\workspace\\ehcachetest\\ehcache.xml";
        CacheManager manager = new CacheManager(fileName);
        String[] names = manager.getCacheNames();
        for (String name : names) {
            System.out.println(name);
        }
        Cache cache = manager.getCache(names[0]);
        cache.put(new Element("key1", "values1"));
        Element element = cache.get("key1");
        System.out.println(element.getValue());
        manager.shutdown();
    }
}

Object Reuse (对象复用)

Object pools keep frequently used instances (e.g., threads, JDBC connections) ready for reuse, avoiding costly creation and destruction. Examples: thread pools, database connection pools such as C3P0 or Proxool.

<Resource name="jdbc/dbsource"
          type="com.mchange.v2.c3p0.ComboPooledDataSource"
          maxPoolSize="50" minPoolSize="5" acquireIncrement="2" initialPoolSize="10" maxIdleTime="60"
          factory="org.apache.naming.factory.BeanFactory"
          user="xxxx" password="xxxx"
          driverClass="oracle.jdbc.driver.OracleDriver"
          jdbcUrl="jdbc:oracle:thin:@192.168.x.x:1521:orcl"
          idleConnectionTestPeriod="10" />

Spring configuration for C3P0 can be defined similarly.

Space‑Time Trade‑off (计算方式转换)

Sometimes sacrificing CPU cycles saves memory (time‑for‑space) or vice‑versa. Example of swapping two variables without a temporary variable:

a = a + b;
b = a - b;
a = a - b;

Unsigned byte handling in Java using bit‑masking:

public class UnsignedByte {
    public short getValue(byte i) { return (short)(i & 0xff); }
    public byte toUnsignedByte(short i) { return (byte)(i & 0xff); }
}

A space‑to‑time sorting algorithm demonstrates the principle by using an auxiliary array indexed by value, eliminating comparisons:

public static void spaceTotime(int[] array) {
    int max = array[0];
    for (int v : array) if (v > max) max = v;
    int[] temp = new int[max + 1];
    for (int v : array) temp[v] = v;
    int j = 0;
    for (int i = 0; i <= max; i++) if (temp[i] > 0) array[j++] = temp[i];
}

Conclusion

The article covers several optimization techniques for high‑throughput Java systems, including buffering, caching, object‑reuse pools, and space‑time trade‑offs, each validated with code samples and performance measurements. Readers should evaluate these methods against their own workloads to choose the most effective solutions.

Author: IBM – Zhou Mingyao Translation source: https://www.ibm.com/developerworks/cn/java/j-lo-system-design-optimization/