Implementing Fast File Chunk Upload with Spring Boot and MinIO

In modern web applications, uploading large files (videos, audio, documents) can be inefficient; splitting files into smaller chunks and uploading them in parallel greatly improves speed and reliability.

1. Introduction

The article demonstrates a fast file‑slice upload solution built with Spring Boot and MinIO, which divides large files into small parts, uploads them concurrently, and merges them on the server.

2. File Slice Upload Overview

File slice upload splits a large file into multiple fragments, uploads them via parallel requests, and reassembles them on the server, mitigating network instability and upload interruptions.

3. Technology Selection

3.1 Spring Boot

Spring Boot provides a lightweight, rapid‑development framework for modern Java applications.

3.2 MinIO

MinIO is an open‑source, S3‑compatible object storage server offering high performance and high availability.

4. Setting Up the Spring Boot Project

Generate a basic project with Spring Initializr, including spring-boot-starter-web and spring-boot-starter-thymeleaf dependencies, and add the MinIO Java client.

<!-- pom.xml -->
<dependencies>
    <!-- Spring Boot Web -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- Thymeleaf template engine -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-thymeleaf</artifactId>
    </dependency>

    <!-- MinIO Java client -->
    <dependency>
        <groupId>io.minio</groupId>
        <artifactId>minio</artifactId>
        <version>8.3.3</version>
    </dependency>
</dependencies>

5. Integrating MinIO

5.1 Configuration in application.properties

# MinIO configuration
minio.endpoint=http://localhost:9000
minio.accessKey=minioadmin
minio.secretKey=minioadmin
minio.bucketName=mybucket

5.2 MinIO Configuration Class

import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class MinioConfig {
    @Value("${minio.endpoint}")
    private String endpoint;
    @Value("${minio.accessKey}")
    private String accessKey;
    @Value("${minio.secretKey}")
    private String secretKey;

    @Bean
    public MinioClient minioClient() {
        return MinioClient.builder()
                .endpoint(endpoint)
                .credentials(accessKey, secretKey)
                .build();
    }
}

6. File Slice Upload Implementation

6.1 Controller Layer

import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;

@RestController
@RequestMapping("/file")
public class FileController {
    @Autowired
    private MinioClient minioClient;
    @Value("${minio.bucketName}")
    private String bucketName;

    @PostMapping("/upload")
    public String upload(@RequestParam("file") MultipartFile file) {
        // implement file slice upload logic
        return "Upload success!";
    }
}

6.2 Service Layer

import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;

@Service
public class FileService {
    @Autowired
    private MinioClient minioClient;
    @Value("${minio.bucketName}")
    private String bucketName;

    public void uploadFile(String objectName, MultipartFile file) throws Exception {
        // implement file slice upload logic
    }
}

6.3 Slice Upload Logic

public void uploadFile(String objectName, MultipartFile file) throws Exception {
    InputStream inputStream = file.getInputStream();
    long size = file.getSize();
    long chunkSize = 5 * 1024 * 1024; // 5 MB per chunk
    long offset = 0;
    while (offset < size) {
        long currentChunkSize = Math.min(chunkSize, size - offset);
        byte[] chunk = new byte[(int) currentChunkSize];
        inputStream.read(chunk);
        minioClient.putObject(
            PutObjectArgs.builder()
                .bucket(bucketName)
                .object(objectName)
                .stream(new ByteArrayInputStream(chunk), currentChunkSize, -1)
                .build()
        );
        offset += currentChunkSize;
    }
    inputStream.close();
}

6.4 Merge Logic

@PostMapping("/merge")
public String merge(@RequestParam String objectName) {
    try {
        fileService.mergeFile(objectName);
        return "Merge success!";
    } catch (Exception e) {
        e.printStackTrace();
        return "Merge failed!";
    }
}

public void mergeFile(String objectName) throws Exception {
    Iterable
parts = minioClient.listObjects(bucketName, objectName);
    for (io.minio.messages.Item part : parts) {
        String partName = part.objectName();
        minioClient.copyObject(
            CopyObjectArgs.builder()
                .source(bucketName, partName)
                .destination(bucketName, objectName)
                .build()
        );
    }
    for (io.minio.messages.Item part : parts) {
        minioClient.removeObject(bucketName, part.objectName());
    }
}

7. Front‑End Page (Thymeleaf)

<!DOCTYPE html>
<html lang="en" xmlns:th="http://www.thymeleaf.org">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Upload</title>
</head>
<body>
    <form id="uploadForm" action="/file/upload" method="post" enctype="multipart/form-data">
        <input type="file" name="file" id="file" />
        <input type="submit" value="Upload" />
    </form>
    <div id="progress" style="display:none;">
        <progress id="progressBar" max="100" value="0"></progress>
        <span id="percentage">0%</span>
    </div>
    <script>
        document.getElementById('uploadForm').addEventListener('submit', function(event) {
            event.preventDefault();
            var file = document.getElementById('file').files[0];
            if (!file) { alert('Please choose a file.'); return; }
            var formData = new FormData();
            formData.append('file', file);
            var xhr = new XMLHttpRequest();
            xhr.open('POST', '/file/upload', true);
            xhr.upload.onprogress = function(e) {
                if (e.lengthComputable) {
                    var percent = Math.round((e.loaded / e.total) * 100);
                    document.getElementById('progressBar').value = percent;
                    document.getElementById('percentage').innerText = percent + '%';
                }
            };
            xhr.onload = function() { document.getElementById('progress').style.display = 'none'; alert('Upload success!'); };
            xhr.onerror = function() { alert('Upload failed!'); };
            xhr.send(formData);
            document.getElementById('progress').style.display = 'block';
        });
    </script>
</body>
</html>

8. Performance Optimizations & Extensions

Concurrent uploading using multithreading or asynchronous tasks to increase throughput.

Distributed deployment to spread storage and application load across multiple servers.

Resumable uploads to continue after interruptions.

Access‑policy based permission control in MinIO for security.

9. Conclusion

The guide shows how to combine Spring Boot and MinIO to realize a fast, reliable file‑chunk upload mechanism, improving upload speed and user experience. By applying the discussed optimizations and extensions, developers can build a robust file upload service suitable for production environments.