Implementation and Principles of HTTP Range (Breakpoint) File Download in Java

The article describes the business background of the Vivo platform, which requires downloading various files (APK, images, etc.) for both C‑end users (app store, game center) and B‑end users (open platform). It outlines the challenges such as network instability, URL redirection, large files, remote file changes, and local file deletions, and emphasizes the need for fast, fault‑tolerant download mechanisms.

To address these challenges, a breakpoint (range) download solution is proposed. The solution includes:

Choosing between direct download, single‑threaded breakpoint download, and multi‑threaded breakpoint download based on file size.

Using a "group mode" (fixed maximum number of threads) to keep server thread count controllable.

Providing fault‑tolerance through automatic retry tasks and manual re‑trigger via a management console.

Performing integrity verification after download.

Issuing alerts when repeated download attempts still fail.

The core of breakpoint download relies on the HTTP Range header. HTTP/1.1 introduced the Accept‑Ranges response header, enabling partial content responses (status 206). The article presents a table of HTTP versions and their support for range requests.

To determine if a URL supports range requests, the following curl command can be used:

curl -I url

If the response contains Accept‑Ranges: bytes , the server supports range requests; otherwise it does not.

Two example curl commands illustrate single‑range and multi‑range requests and their responses (status 206, Content‑Type: multipart/byteranges , etc.).

When a remote file changes during a download, the ETag and Last‑Modified headers can be used to detect the change. An unchanged ETag yields a 304 Not Modified response, while a modified ETag results in a 200 OK response, indicating that the previous range request is no longer valid.

Implementation details in Java are provided:

/**
 * 获取连接
 */
private static HttpURLConnection getHttpUrlConnection(String netUrl) throws Exception {
    URL url = new URL(netUrl);
    HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
    // 设置超时间为3秒
    httpURLConnection.setConnectTimeout(3 * 1000);
    // 防止屏蔽程序抓取而返回403错误
    httpURLConnection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)");
    return httpURLConnection;
}

/**
 * 判断连接是否支持断点下载
 */
private static boolean isSupportRange(String netUrl) throws Exception {
    HttpURLConnection httpURLConnection = getHttpUrlConnection(netUrl);
    String acceptRanges = httpURLConnection.getHeaderField("Accept-Ranges");
    if (StringUtils.isEmpty(acceptRanges)) {
        return false;
    }
    return "bytes".equalsIgnoreCase(acceptRanges);
}

Single‑threaded breakpoint download splits the file into N segments and sequentially appends each segment to a local file:

private static void segmentDownload(int totalFileSize, String netUrl, int N) throws Exception {
    String localFilePath = "F:\\test_single_thread.txt";
    int eachFileSize = totalFileSize / N;
    for (int i = 1; i <= N; i++) {
        long start = new File(localFilePath).length();
        long end = (i == 1) ? eachFileSize : (i == N) ? totalFileSize : eachFileSize * i;
        appendFile(netUrl, new File(localFilePath), start, end);
        System.out.println(String.format("我是第%s次下载，下载片段范围start=%s,end=%s", i, start, end));
    }
    System.out.println("本地文件大小：" + new File(localFilePath).length());
}

/**
 * 文件尾部追加
 */
private static void appendFile(String netUrl, File localFile, long start, long end) throws Exception {
    HttpURLConnection httpURLConnection = getHttpUrlConnection(netUrl);
    httpURLConnection.setRequestProperty("Range", "bytes=" + start + "-" + end);
    InputStream inputStream = httpURLConnection.getInputStream();
    FileOutputStream fos = FileUtils.openOutputStream(localFile, true);
    IOUtils.copy(inputStream, fos);
    closeHttpUrlConnection(httpURLConnection);
}

Multi‑threaded download uses a thread pool and RandomAccessFile to write each segment directly to its correct offset:

private static void groupDownload(String netUrl, int totalFileSize, int N) throws Exception {
    CountDownLatch countDownLatch = new CountDownLatch(N);
    String localFilePath = "F:\\test_multiple_thread.txt";
    int groupSize = totalFileSize / N;
    for (int i = 1; i <= N; i++) {
        int start, end;
        if (i == 1) { start = 0; end = groupSize; }
        else if (i == N) { start = groupSize * (i - 1) + 1; end = totalFileSize; }
        else { start = groupSize * (i - 1) + 1; end = groupSize * i; }
        System.out.println(String.format("线程%s分配区间范围start=%s, end=%s", i, start, end));
        downloadAndMerge(i, netUrl, localFilePath, start, end, countDownLatch);
    }
    countDownLatch.await();
    validateCompleteness(localFilePath, netUrl);
}

private static void downloadAndMerge(int threadNum, String netUrl, String localFilePath, int start, int end, CountDownLatch latch) {
    threadPoolExecutor.execute(() -> {
        try {
            HttpURLConnection conn = getHttpUrlConnection(netUrl);
            conn.setRequestProperty("Range", "bytes=" + start + "-" + end);
            InputStream in = conn.getInputStream();
            RandomAccessFile raf = new RandomAccessFile(localFilePath, "rw");
            raf.seek(start);
            byte[] buffer = new byte[1024 * 10];
            int len;
            while ((len = in.read(buffer)) != -1) {
                raf.write(buffer, 0, len);
            }
            closeHttpUrlConnection(conn);
            System.out.println(String.format("下载完成时间%s, 线程：%s, 下载完成: start=%s, end = %s", System.currentTimeMillis(), threadNum, start, end));
        } catch (Exception e) {
            System.out.println(String.format("片段下载异常：线程：%s, start=%s, end = %s", threadNum, start, end));
            e.printStackTrace();
        }
        latch.countDown();
    });
}

After downloading, integrity is verified by comparing the file’s MD5 hash with the server’s ETag:

private static void validateCompleteness(String localFilePath, String netUrl) throws Exception {
    File file = new File(localFilePath);
    InputStream data = new FileInputStream(file);
    String md5 = DigestUtils.md5Hex(data);
    HttpURLConnection conn = getHttpUrlConnection(netUrl);
    String etag = conn.getHeaderField("Etag");
    if (etag != null && etag.toUpperCase().contains(md5.toUpperCase())) {
        System.out.println(String.format("本地文件和远程文件一致，md5 = %s, Etag = %s", md5.toUpperCase(), etag));
    } else {
        System.out.println(String.format("本地文件和远程文件不一致，md5 = %s, Etag = %s", md5.toUpperCase(), etag));
    }
}

The article concludes that breakpoint download can significantly improve download speed, but it should be applied judiciously—if the network is stable and files are relatively small, the added complexity may not be worthwhile.