Master Fast File Uploads: Instant, Chunked, and Resumable Strategies in Java

File upload is a common challenge; simple byte‑stream uploads work for small files but become inefficient for large files, especially when interruptions force a restart.

Instant Upload (秒传)

The server first computes the file's MD5; if a matching file already exists, it returns a new URL pointing to the existing data, effectively skipping the upload. Changing the MD5 (e.g., by modifying the file content) disables this shortcut.

Chunked Upload (分片上传)

Large files are divided into equal‑size parts (chunks) that are uploaded separately and later reassembled on the server. This approach is ideal for big files and unreliable network conditions.

Resumable Upload (断点续传)

Similar to chunked upload, but if an upload is interrupted, the client records which chunks succeeded and resumes from the last incomplete chunk, avoiding a full restart.

Core Implementation Logic

Redis stores the upload status using the file's MD5 as the key. When the flag is true, the server performs instant upload; when false, it records each chunk's path with a key composed of the MD5 plus a fixed prefix.

Implementation Steps

Standard workflow: split the file, initialize a chunked upload task to obtain a unique ID, send chunks (serially or in parallel), and finally merge them into the original file.

Custom workflow used in this article: the client sends the chunk index and size; the server creates a .conf file whose length equals the total number of chunks. Each uploaded chunk writes a byte value 127 at its position; unuploaded chunks remain 0. The server uses RandomAccessFile or MappedByteBuffer to write data at the correct offset.

Backend Code Examples

RandomAccessFile implementation:

@UploadMode(mode = UploadModeEnum.RANDOM_ACCESS)
@Slf4j
public class RandomAccessUploadStrategy extends SliceUploadTemplate {
    @Autowired
    private FilePathUtil filePathUtil;

    @Value("${upload.chunkSize}")
    private long defaultChunkSize;

    @Override
    public boolean upload(FileUploadRequestDTO param) {
        RandomAccessFile accessTmpFile = null;
        try {
            String uploadDirPath = filePathUtil.getPath(param);
            File tmpFile = super.createTmpFile(param);
            accessTmpFile = new RandomAccessFile(tmpFile, "rw");
            long chunkSize = Objects.isNull(param.getChunkSize()) ? defaultChunkSize * 1024 * 1024 : param.getChunkSize();
            long offset = chunkSize * param.getChunk();
            accessTmpFile.seek(offset);
            accessTmpFile.write(param.getFile().getBytes());
            boolean isOk = super.checkAndSetUploadProgress(param, uploadDirPath);
            return isOk;
        } catch (IOException e) {
            log.error(e.getMessage(), e);
        } finally {
            FileUtil.close(accessTmpFile);
        }
        return false;
    }
}

MappedByteBuffer implementation:

@UploadMode(mode = UploadModeEnum.MAPPED_BYTEBUFFER)
@Slf4j
public class MappedByteBufferUploadStrategy extends SliceUploadTemplate {
    @Autowired
    private FilePathUtil filePathUtil;

    @Value("${upload.chunkSize}")
    private long defaultChunkSize;

    @Override
    public boolean upload(FileUploadRequestDTO param) {
        RandomAccessFile tempRaf = null;
        FileChannel fileChannel = null;
        MappedByteBuffer mappedByteBuffer = null;
        try {
            String uploadDirPath = filePathUtil.getPath(param);
            File tmpFile = super.createTmpFile(param);
            tempRaf = new RandomAccessFile(tmpFile, "rw");
            fileChannel = tempRaf.getChannel();
            long chunkSize = Objects.isNull(param.getChunkSize()) ? defaultChunkSize * 1024 * 1024 : param.getChunkSize();
            long offset = chunkSize * param.getChunk();
            byte[] fileData = param.getFile().getBytes();
            mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, offset, fileData.length);
            mappedByteBuffer.put(fileData);
            boolean isOk = super.checkAndSetUploadProgress(param, uploadDirPath);
            return isOk;
        } catch (IOException e) {
            log.error(e.getMessage(), e);
        } finally {
            FileUtil.freedMappedByteBuffer(mappedByteBuffer);
            FileUtil.close(fileChannel);
            FileUtil.close(tempRaf);
        }
        return false;
    }
}

Template class that provides common utilities (creating temporary files, tracking progress in the .conf file, persisting status to Redis, and renaming the final file) is also included in the source.

Conclusion

Successful chunked uploads require the front‑end and back‑end to agree on chunk size; a dedicated file server (e.g., FastDFS, HDFS) is typically needed. For simpler upload/download needs, an object storage service such as Alibaba Cloud OSS can be used, though it may not suit heavy delete/modify workloads.