How to Speed Up Java File Compression: From 30 Seconds to 1 Second with NIO
This article walks through a real‑world Java file‑compression task, shows why a naïve implementation takes 30 seconds for a 20 MB batch, and demonstrates five progressive optimizations—BufferedInputStream, NIO Channel, memory‑mapped files, Pipe, and code snippets—that cut the time down to about one second while explaining the underlying I/O mechanisms.
A requirement arose to receive ten photos from the front‑end, compress them into a zip archive, and stream the result. The initial naïve implementation using FileInputStream read one byte at a time, resulting in a 30 second processing time for a 20 MB batch.
public static void zipFileNoBuffer() {
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile))) {
long beginTime = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
try (InputStream input = new FileInputStream(JPG_FILE)) {
zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));
int temp = 0;
while ((temp = input.read()) != -1) {
zipOut.write(temp);
}
}
}
printInfo(beginTime);
} catch (Exception e) {
e.printStackTrace();
}
}Result:
fileSize:20M consum time:29599ms.
First optimization – BufferedInputStream (≈2 seconds)
Wrapping the FileInputStream with a BufferedInputStream reduces the number of native read calls because data is read in larger blocks (default 8 KB).
public static void zipFileBuffer() {
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(zipOut)) {
long beginTime = System.currentTimeMillis();
for (int i = 0; i < 10; i++) {
try (BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(JPG_FILE))) {
zipOut.putNextEntry(new ZipEntry(FILE_NAME + i));
int temp = 0;
while ((temp = bufferedInputStream.read()) != -1) {
bufferedOutputStream.write(temp);
}
}
}
printInfo(beginTime);
} catch (Exception e) {
e.printStackTrace();
}
}Result:
fileSize:20M consum time:1808ms.
Second optimization – NIO Channel with transferTo (≈1.4 seconds)
Using a FileChannel and its transferTo method lets the operating system move bytes directly from the source channel to the destination channel, bypassing user‑space copies.
public static void zipFileChannel() {
long beginTime = System.currentTimeMillis();
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {
for (int i = 0; i < 10; i++) {
try (FileChannel fileChannel = new FileInputStream(JPG_FILE).getChannel()) {
zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE));
fileChannel.transferTo(0, FILE_SIZE, writableByteChannel);
}
}
printInfo(beginTime);
} catch (Exception e) {
e.printStackTrace();
}
}Result:
fileSize:20M consum time:1416ms.
Third optimization – Memory‑mapped file (≈1.3 seconds)
Mapping the source file into memory with MappedByteBuffer allows the kernel to serve data directly from the page cache.
public static void zipFileMap() {
long beginTime = System.currentTimeMillis();
File zipFile = new File(ZIP_FILE);
try (ZipOutputStream zipOut = new ZipOutputStream(new FileOutputStream(zipFile));
WritableByteChannel writableByteChannel = Channels.newChannel(zipOut)) {
for (int i = 0; i < 10; i++) {
zipOut.putNextEntry(new ZipEntry(i + SUFFIX_FILE));
MappedByteBuffer mappedByteBuffer = new RandomAccessFile(JPG_FILE_PATH, "r").getChannel()
.map(FileChannel.MapMode.READ_ONLY, 0, FILE_SIZE);
writableByteChannel.write(mappedByteBuffer);
}
printInfo(beginTime);
} catch (Exception e) {
e.printStackTrace();
}
}Result:
fileSize:20M consum time:1305ms.
Fourth optimization – Pipe with asynchronous task (≈1 second)
Using a Pipe separates reading and writing into two threads, allowing the producer to stream data directly into the zip output.
public static void zipFilePip() {
long beginTime = System.currentTimeMillis();
try (WritableByteChannel out = Channels.newChannel(new FileOutputStream(ZIP_FILE))) {
Pipe pipe = Pipe.open();
CompletableFuture.runAsync(() -> runTask(pipe));
ReadableByteChannel readableByteChannel = pipe.source();
ByteBuffer buffer = ByteBuffer.allocate((int) FILE_SIZE * 10);
while (readableByteChannel.read(buffer) >= 0) {
buffer.flip();
out.write(buffer);
buffer.clear();
}
} catch (Exception e) {
e.printStackTrace();
}
printInfo(beginTime);
}
public static void runTask(Pipe pipe) {
try (ZipOutputStream zos = new ZipOutputStream(Channels.newOutputStream(pipe.sink()));
WritableByteChannel out = Channels.newChannel(zos)) {
for (int i = 0; i < 10; i++) {
zos.putNextEntry(new ZipEntry(i + SUFFIX_FILE));
FileChannel jpgChannel = new FileInputStream(new File(JPG_FILE_PATH)).getChannel();
jpgChannel.transferTo(0, FILE_SIZE, out);
jpgChannel.close();
}
} catch (Exception e) {
e.printStackTrace();
}
}Result:
fileSize:20M consum time:~1000ms(the exact figure varies with the environment).
Summary
Reading a file byte‑by‑byte via FileInputStream is extremely slow because each byte triggers a native system call.
Wrapping the stream with BufferedInputStream reduces system calls by buffering data in memory.
Java NIO channels and transferTo enable zero‑copy transfers, moving data directly between kernel buffers.
Memory‑mapped files and Pipes further exploit OS‑level optimizations, achieving sub‑second compression for a 20 MB batch.
Understanding these I/O mechanisms helps you choose the most efficient approach for large‑scale file processing.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Programmer DD
A tinkering programmer and author of "Spring Cloud Microservices in Action"
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
