How to Build a Scalable Chunked File Upload Service with Spring Boot and MinIO

Background

In real‑world development, large file uploads such as 2 GB videos, backend ZIP packages, or interrupted network transfers are common. Using a single MultipartFile upload leads to poor front‑end experience, high backend memory usage, and limited scalability.

Design Goals

Chunked upload: split large files into small parts.

Resumable upload: continue from the last successful chunk after a network failure.

High availability: support distributed deployment without relying on local files.

Extensibility: compatible with object storage such as MinIO or OSS.

CDN acceleration: enable file distribution and download optimization.

Technology Stack

Spring Boot – main framework.

MinIO – object storage (compatible with OSS).

MySQL – store upload status and chunk records.

Redis – optional cache for concurrent chunk state.

CDN – file distribution (e.g., Qiniu, Alibaba Cloud).

Module Diagram

Core Table Schema

CREATE TABLE file_upload_record (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    file_md5 VARCHAR(64) NOT NULL,
    file_name VARCHAR(255),
    total_chunks INT,
    uploaded_chunks INT DEFAULT 0,
    is_complete BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Core API Design

Check if file already uploaded

@GetMapping("/upload/check")
public ResponseEntity<?> checkFile(@RequestParam String fileMd5) {
    FileUploadRecord record = recordRepository.findByFileMd5(fileMd5);
    if (record != null && record.getIsComplete()) {
        return ResponseEntity.ok(Map.of("uploaded", true, "url", getFileUrl(fileMd5)));
    }
    return ResponseEntity.ok(Map.of("uploaded", false));
}

Upload a chunk

@PostMapping("/upload/chunk")
public ResponseEntity<?> uploadChunk(@RequestParam String fileMd5,
                                     @RequestParam int chunkIndex,
                                     @RequestParam MultipartFile filePart) throws IOException {
    String objectName = String.format("upload/%s/%d.part", fileMd5, chunkIndex);
    // upload to MinIO
    minioClient.putObject(PutObjectArgs.builder()
        .bucket(bucketName)
        .object(objectName)
        .stream(filePart.getInputStream(), filePart.getSize(), -1)
        .contentType(filePart.getContentType())
        .build());
    // update DB status
    recordService.markChunkUploaded(fileMd5, chunkIndex);
    return ResponseEntity.ok("Chunk uploaded");
}

Merge chunks into final file

@PostMapping("/upload/merge")
public ResponseEntity<?> mergeChunks(@RequestParam String fileMd5,
                                    @RequestParam int totalChunks) throws Exception {
    String finalObjectName = "upload/" + fileMd5 + ".final";
    List<ComposeSource> sources = new ArrayList<>();
    for (int i = 0; i < totalChunks; i++) {
        sources.add(ComposeSource.builder()
            .bucket(bucketName)
            .object(String.format("upload/%s/%d.part", fileMd5, i))
            .build());
    }
    minioClient.composeObject(ComposeObjectArgs.builder()
        .bucket(bucketName)
        .object(finalObjectName)
        .sources(sources)
        .build());
    // mark upload complete
    recordService.markComplete(fileMd5);
    return ResponseEntity.ok(Map.of("url", getFileUrl(fileMd5)));
}

Key Logic Details

Using fileMd5 as the unique identifier ensures file deduplication and enables resumable uploads; the client can pre‑compute the MD5 and query which chunks are already stored.

MinIO’s composeObject merges multiple objects server‑side, similar to S3 multipart upload, reducing network overhead.

If the network breaks, the client checks uploaded chunks via /upload/check and only sends missing parts.

Extensibility Design

A storage interface abstracts the underlying object store, allowing implementations for MinIO, OSS, FastDFS, etc., thus reducing coupling.

public interface FileStorageService {
    void uploadChunk(String objectName, InputStream stream, long size);
    void mergeChunks(String finalName, List<String> chunkNames);
    String getFileUrl(String objectName);
}

Performance & High‑Availability Optimizations

Upload rate limiting via Nginx and front‑end batch uploads.

Idempotent endpoints controlled by Redis caching of chunk status.

Asynchronous merge with a task scheduler to reduce merge latency.

Object storage enables multi‑node shared access for distributed deployments.

CDN URL Generation

public String getFileUrl(String fileMd5) {
    return cdnDomain + "/upload/" + fileMd5 + ".final";
}

Conclusion

The presented solution delivers a high‑availability, scalable large‑file upload interface that supports chunked and resumable uploads, leverages MinIO/S3 object storage, integrates CDN distribution, and can be easily extended to other storage providers.