Master Chunked File Uploads with WebUploader and Spring Boot
This guide explains how to use Baidu's WebUploader component together with a Spring Boot backend to implement large file uploads by splitting files into chunks, supporting breakpoint resume, MD5‑based instant upload, and also provides a parallel download client using HTTP range requests.
WebUploader is a modern file upload component developed by Baidu WebFE, primarily based on HTML5 with optional Flash support.
Large File Upload
Implementation ideas:
Chunking: split a large file into multiple small fragments according to a custom buffer size.
Breakpoint resume: each fragment is named uniquely; when a transfer is interrupted, existing fragments are detected and skipped, while unfinished fragments are re‑uploaded.
Merge: after the last fragment is uploaded, a merge method concatenates the fragments in order, handling possible out‑of‑order arrivals with a while loop.
Instant upload: calculate a unique MD5 for the whole file; if the MD5 already exists on the server, the file is not uploaded again.
Create a Spring Boot project and add the following Maven dependencies:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>Verify the environment with a simple controller:
@Controller
public class UploadController {
@RequestMapping("/up")
@ResponseBody
public String upload(HttpServletRequest request, HttpServletResponse response) {
return "setup successful";
}
}Front‑end page code (HTML + JavaScript) creates an upload container, a list area, and a button, then configures WebUploader:
<body>
<div id="upload-container"><span>Upload</span></div>
<div id="upload-list"></div>
<button id="picker">Click to Upload</button>
</body>
<script>
$("#upload-container").click(function(){
$("#picker").find('input').click();
});
var uploader = WebUploader.create({
auto: true,
swf: 'Uploader.swf',
server: 'http://localhost:8080/upload',
dnd: '#upload-container',
pick: '#picker',
multiple: true,
chunked: true,
threads: 20,
method: 'POST',
fileSizeLimit: 1024*1024*1024*10,
fileSingleSizeLimit: 1024*1024*1024,
fileVal: 'upload'
});
uploader.on('beforeFileQueued', function(file){ console.log(file); });
uploader.on('fileQueued', function(file){
console.log(file.ext, file.size, file.name);
var html = '<div class="upload-item"><span>File: '+file.name+'</span>'+
'<span data-file_id="'+file.id+'" class="btn-delete">Delete</span>'+
'<span data-file_id="'+file.id+'" class="btn-retry">Retry</span>'+
'<div class="percentage '+file.id+'" style="width:0%;"></div></div>';
$('#upload-list').append(html);
uploader.md5File(file).then(function(val){ console.log('md5 result', val); });
});
</script>Server‑side breakpoint‑resume handling parses the multipart request, stores each chunk under a temporary name, and merges them when the last chunk arrives:
@Controller
public class UploadController {
private static final String UTF8 = "utf-8";
@RequestMapping("/up")
@ResponseBody
public void upload(HttpServletRequest request, HttpServletResponse response) throws Exception {
response.setCharacterEncoding(UTF8);
Integer chunk = null, chunks = null; String name = null; String path = "D:\\file";
BufferedOutputStream os = null;
try {
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(1024);
factory.setRepository(new File(path));
ServletFileUpload upload = new ServletFileUpload(factory);
upload.setFileSizeMax(5L*1024*1024*1024);
upload.setSizeMax(10L*1024*1024*1024);
List<FileItem> items = upload.parseRequest(request);
for (FileItem item : items) {
if (item.isFormField()) {
if ("chunk".equals(item.getFieldName())) chunk = Integer.parseInt(item.getString(UTF8));
if ("chunks".equals(item.getFieldName())) chunks = Integer.parseInt(item.getString(UTF8));
if ("name".equals(item.getFieldName())) name = item.getString(UTF8);
}
}
for (FileItem item : items) {
if (!item.isFormField()) {
String tempFileName = name;
if (name != null) {
if (chunk != null) tempFileName = chunk + "_" + name;
File tempFile = new File(path, tempFileName);
if (!tempFile.exists()) item.write(tempFile);
}
}
}
if (chunk != null && chunk.intValue() == chunks.intValue() - 1) {
File finalFile = new File(path, name);
os = new BufferedOutputStream(new FileOutputStream(finalFile));
for (int i = 0; i < chunks; i++) {
File part = new File(path, i + "_" + name);
while (!part.exists()) Thread.sleep(100);
byte[] bytes = FileUtils.readFileToByteArray(part);
os.write(bytes);
os.flush();
part.delete();
}
os.flush();
}
response.getWriter().write("upload successful");
} finally {
if (os != null) os.close();
}
}
}File download supports HTTP range requests for partial download, setting headers such as Accept-Range, Content-Range, and streaming the requested byte segment:
@Controller
public class DownLoadController {
private static final String UTF8 = "utf-8";
@RequestMapping("/down")
public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
response.setCharacterEncoding(UTF8);
File file = new File("D:\\File\\a.mp4");
long fSize = file.length();
response.setContentType("application/x-download");
String fileName = URLEncoder.encode(file.getName(), UTF8);
response.addHeader("Content-Disposition", "attachment;filename=" + fileName);
response.setHeader("Accept-Range", "bytes");
response.setHeader("fSize", String.valueOf(fSize));
response.setHeader("fName", fileName);
long pos = 0, last = fSize - 1, sum = 0;
if (request.getHeader("Range") != null) {
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
String range = request.getHeader("Range").replaceAll("bytes=", "");
String[] parts = range.split("-");
if (parts.length == 2) {
pos = Long.parseLong(parts[0].trim());
last = Long.parseLong(parts[1].trim());
if (last > fSize - 1) last = fSize - 1;
} else {
pos = Long.parseLong(range.replaceAll("-", "").trim());
}
}
long rangeLength = last - pos + 1;
String contentRange = new StringBuilder("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
response.setHeader("Content-Range", contentRange);
response.setHeader("Content-Lenght", String.valueOf(rangeLength));
try (BufferedOutputStream os = new BufferedOutputStream(response.getOutputStream());
BufferedInputStream is = new BufferedInputStream(new FileInputStream(file))) {
is.skip(pos);
byte[] buffer = new byte[1024];
int len;
while (sum < rangeLength) {
len = is.read(buffer, 0, (int)Math.min(buffer.length, rangeLength - sum));
sum += len;
os.write(buffer, 0, len);
}
}
}
}A client implementation downloads a file in parallel chunks using a thread pool, writes each chunk to a temporary file, and merges them after all parts are received:
@RestController
public class DownloadClient {
private static final long PER_PAGE = 1024L*1024L*50L;
private static final String DOWN_PATH = "D:\\File";
private ExecutorService pool = Executors.newFixedThreadPool(10);
@RequestMapping("/downloadFile")
public String downloadFile() throws IOException {
FileInfo fileInfo = download(0, 10, -1, null);
if (fileInfo != null) {
long pages = fileInfo.fSize / PER_PAGE;
for (int i = 0; i <= pages; i++) {
pool.submit(new Download(i * PER_PAGE, (i + 1) * PER_PAGE - 1, i, fileInfo.fName));
}
}
return "success";
}
class Download implements Runnable {
long start, end, page; String fName;
Download(long start, long end, long page, String fName) { this.start=start; this.end=end; this.page=page; this.fName=fName; }
@Override
public void run() {
try { download(start, end, page, fName); } catch (IOException e) { e.printStackTrace(); }
}
}
private FileInfo download(long start, long end, long page, String fName) throws IOException {
File file = new File(DOWN_PATH, page + "-" + fName);
if (file.exists() && page != -1 && file.length() == PER_PAGE) return null;
HttpClient client = HttpClients.createDefault();
HttpGet get = new HttpGet("http://127.0.0.1:8080/down");
get.setHeader("Range", "bytes=" + start + "-" + end);
HttpResponse resp = client.execute(get);
String fSize = resp.getFirstHeader("fSize").getValue();
fName = URLDecoder.decode(resp.getFirstHeader("fName").getValue(), "utf-8");
InputStream is = resp.getEntity().getContent();
try (FileOutputStream fos = new FileOutputStream(file)) {
byte[] buf = new byte[1024];
int len;
while ((len = is.read(buf)) != -1) fos.write(buf, 0, len);
}
if (end - Long.valueOf(fSize) > 0) mergeFile(fName, page);
return new FileInfo(Long.valueOf(fSize), fName);
}
private void mergeFile(String fName, long page) throws Exception {
File out = new File(DOWN_PATH, fName);
try (BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(out))) {
for (int i = 0; i <= page; i++) {
File part = new File(DOWN_PATH, i + "-" + fName);
while (!part.exists() || (i != page && part.length() < PER_PAGE)) Thread.sleep(100);
os.write(FileUtils.readFileToByteArray(part));
part.delete();
}
}
}
class FileInfo { long fSize; String fName; FileInfo(long fSize, String fName) { this.fSize = fSize; this.fName = fName; } }
}Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Java High-Performance Architecture
Sharing Java development articles and resources, including SSM architecture and the Spring ecosystem (Spring Boot, Spring Cloud, MyBatis, Dubbo, Docker), Zookeeper, Redis, architecture design, microservices, message queues, Git, etc.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.