How to Build a Robust Multi‑File Parallel Upload System for Mobile Apps

Background

Short‑video apps such as Douyin and Kuaishou allow users to edit videos locally and then upload them; the underlying upload mechanism is the same for any file type.

Overall Solution Design

The upload workflow is divided into three core components:

Task Management

Resumable (秒传) Logic

Chunked Upload

1. Task Management

Each video file is treated as a task. All tasks are stored in an uploadTasks array. When a new task is added, the system checks whether another task is already uploading. If no upload is in progress, the task is started immediately; otherwise it is queued behind existing tasks.

The UploadManager implements the core upload logic, while the surrounding code handles business‑specific concerns.

2. Resumable Logic

Before requesting an upload URL, the client computes the file’s md5 hash and sends it to the server. The server compares the hash with existing files; if a matching file exists, it returns the existing file id and the client skips the actual file transfer, completing the upload through metadata only.

3. Chunked Upload

Each file is split into multiple chunks. A task represents the whole file, while each chunk is uploaded via a multipart/form‑data request. The upload address is dynamically generated and expires, so the client must request a fresh address before each (re)upload.

First upload: call upload.do to obtain the address and file id.

Resume upload: call resume.do with the existing id to get a resume address.

Chunk handling uses a handle method that seeks to the appropriate size offset, reads bytes, and assembles the multipart form data.

The process maintains two arrays: uploadSegments – indexes of pending chunks. successSegments – indexes of successfully uploaded chunks.

When a chunk succeeds, its index moves from uploadSegments to successSegments. Once the counts of both arrays match the total number of chunks, the client notifies the server that the file upload is complete.

Dynamic Concurrency and Retry

The upload module measures network speed and adjusts the number of parallel chunk uploads on the fly. If a chunk fails due to network jitter or server errors, the system retries up to three times based on the error case. Persistent errors (e.g., file format issues) abort the upload with an error message.

Memory‑Peak Management

Uploading large (e.g., 1 GB) video files can cause high memory usage. Using NSURLSessionUploadTask with fromFile streams the file from disk, avoiding the memory spike that occurs with fromData. The implementation writes each chunk to a temporary file, then uploads the file path, keeping memory consumption low even for gigabyte‑size videos.

--boundary
Content-Disposition: form-data; name="parameterName"
parameterValue
--boundary
Content-Disposition: form-data; name="file"; filename="filename"
Content-Type: mime/type
(binary data)
--boundary--

- (NSString *)writeMultipartFormData:(NSData *)data parameters:(NSDictionary *)parameters {
    if (data.length == 0) { return nil; }
    NSMutableData *formData = [NSMutableData data];
    NSData *lineData = ["
" dataUsingEncoding:NSUTF8StringEncoding];
    NSString *boundaryString = [NSString stringWithFormat:@"--%@", Boundary];
    NSData *boundary = [boundaryString dataUsingEncoding:NSUTF8StringEncoding];
    // Append parameters
    [parameters enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL *stop) {
        [formData appendData:boundary];
        [formData appendData:lineData];
        NSString *field = [NSString stringWithFormat:@"Content-Disposition: form-data; name=\"%@\"

%@", key, obj];
        [formData appendData:[field dataUsingEncoding:NSUTF8StringEncoding]];
        [formData appendData:lineData];
    }];
    // Append file data
    [formData appendData:boundary];
    [formData appendData:lineData];
    NSString *fileField = @"Content-Disposition: form-data; name=\"name\"; filename=\"filename\"
Content-Type: mimetype";
    [formData appendData:[fileField dataUsingEncoding:NSUTF8StringEncoding]];
    [formData appendData:lineData];
    [formData appendData:lineData];
    [formData appendData:data];
    [formData appendData:lineData];
    [formData appendData:[[NSString stringWithFormat:@"--%@--
", Boundary] dataUsingEncoding:NSUTF8StringEncoding]];
    NSString *filePath = [NSString stringWithFormat:@"%@/%ld", self.segmentDocumentPath, self.currentIndex];
    BOOL write = [formData writeToFile:filePath atomically:YES];
    return write ? filePath : nil;
}

Exception Handling

Common error cases and their handling:

If memory runs out or the network is unavailable, pause all tasks and persist their state.

For unknown network errors, retry up to three times; if still failing, pause the task.

If the device is in airplane mode or network access is denied, inform the user and pause the task.

Conclusion

The described multi‑file parallel upload architecture—combining task queuing, MD5‑based resumable uploads, chunked multipart streaming, dynamic concurrency, and robust error handling—provides high reliability for large video uploads on mobile platforms.