Why Spring Boot's /tmp Fills Up Even When Streaming Files to MinIO

Problem: /tmp Exhausted Despite Streaming to MinIO

A file upload endpoint crashed the service when several users simultaneously uploaded large videos. Files were streamed directly to MinIO via MultipartFile.getInputStream() with no local Files.write() or transferTo(), yet the container's temporary disk hit 100% usage, causing No space left on device errors across unrelated interfaces.

Root Cause: Multipart Parsing Happens Before Controller

Spring MVC's servlet multipart model parses the request before the controller receives the MultipartFile. The servlet container (Tomcat/Jetty) writes parts exceeding file-size-threshold (default 0) to a temporary directory ( spring.servlet.multipart.location, defaulting to system /tmp). Thus the flow is actually:

Browser → Servlet multipart parser → Temporary file → Controller gets MultipartFile → Read temp file → MinIO

Not the assumed:

Browser → Spring Boot → InputStream → MinIO

Concurrent Large Uploads Multiply Temporary Disk Usage

With max-file-size: 2GB and a 20 GB temporary disk, 10 concurrent 2 GB uploads require ~20 GB before any reach MinIO. Slow client networks prolong temporary file retention. The real limit is not single-file size but concurrent in-flight uploads consuming temporary space.

Configuration Fixes: Isolate Temporary Directory

Explicitly set a dedicated location to avoid contaminating system /tmp:

spring:
  servlet:
    multipart:
      location: /data/upload-tmp
      max-file-size: 2GB
      max-request-size: 2GB
      file-size-threshold: 0

In Kubernetes, mount a separate volume for /data/upload-tmp to isolate upload traffic from the container root filesystem.

Architectural Fix: Direct Browser-to-Object-Storage Upload

If the final destination is S3/MinIO/OSS, bypass the Java service entirely for data transfer:

Frontend requests POST /api/files/upload-session.

Java validates permissions, file type, size, and generates a presigned URL (e.g., 15-minute expiry).

Frontend uploads directly to object storage.

Frontend calls POST /api/files/complete with object metadata.

Java verifies object existence, size, content-type, and session ownership before creating business record.

Example session creation:

public UploadSession createUploadSession(CreateUploadRequest request) {
    checkPermission();
    validateFileMeta(request);
    String objectKey = objectKeyGenerator.generate(currentUserId(), request.fileName());
    String uploadUrl = objectStorageService.createPresignedUploadUrl(objectKey, Duration.ofMinutes(15));
    return new UploadSession(objectKey, uploadUrl);
}

Security: Never Trust Client-Provided Metadata

On completion, re-fetch object metadata from storage and verify:

Object belongs to the upload session

File actually exists

Session not expired

Size matches policy

Content-Type allowed

Also validate file content via magic numbers, not just extension or Content-Type header.

Large Files: Use Multipart Upload for Resumability

For multi-GB files, initialize a multipart upload in object storage, issue presigned URLs per part, and let the browser upload parts directly. Java only manages session, part authorization, and final completion. This survives network interruptions and Java pod restarts.

If Java Must Handle Data: Enforce Concurrency Quotas

When direct upload is impossible (intranet, compliance, streaming processing), limit concurrent uploads per instance:

private final Semaphore uploadSlots = new Semaphore(4);

public FileResponse upload(MultipartFile file) {
    if (!uploadSlots.tryAcquire()) throw new TooManyUploadsException();
    try { return doUpload(file); }
    finally { uploadSlots.release(); }
}

Treat temporary disk like a connection pool: if each upload may consume 2 GB and safe capacity is 10 GB, allow at most 4-5 concurrent uploads.

Cleanup Guarantees Are Not Enough

Temporary files are deleted after request completion, but disk exhaustion occurs during the request. Peak concurrent usage matters, not eventual cleanup. Monitor real-time temporary directory usage with alerts at 70% (warning) and 85% (critical).

Handle Client Disconnects Gracefully

If the client aborts mid-upload, any custom temporary files must be cleaned in finally blocks. Implement a background orphan-file cleaner that only removes files exceeding a safe age, not a blunt rm -rf /tmp/*.

Re-evaluate file-size-threshold

Raising file-size-threshold to keep files in memory trades disk pressure for JVM OOM risk. Ten concurrent 500 MB uploads = 5 GB heap pressure. The fundamental question remains: why must gigabytes pass through the Java process at all?

Final Architecture After Refactor

Old:

Browser → Spring Boot Multipart → Pod Temp Disk → Java InputStream → MinIO

New:

Browser → Request Java for Credentials → MinIO → Notify Java Completion → Java Verifies Object → Create Business Record

Java now handles only metadata (bytes/KB), eliminating temporary disk pressure. Added: size limits, session expiry, object-key permissions, completion verification, multipart upload, and failure cleanup.

Going forward, any @PostMapping("/upload") MultipartFile triggers the question: How much resource has the server already spent before this parameter arrives? The best optimization for large files to object storage is to never let them touch the Java service.