Spring Boot & MinIO: Building an Enterprise File Platform with Multipart Upload & Multi-tenancy

1. Business Pain Points & Object Storage Centralization

Early on each business line built its own file service using NAS, NFS, or local disks. Once data exceeded tens of millions and concurrency grew, problems surfaced: inconsistent APIs, scattered authentication, cold data occupying SSDs, downtime required for scaling, and no unified access logs for compliance audits. Centralizing into an object storage platform became an operational necessity. Object storage natively supports HTTP/RESTful, separates metadata from data, and scales horizontally. MinIO is fully S3-compatible, has mature erasure coding, and integrates with the Spring Boot ecosystem to quickly converge into an observable, governable storage foundation.

2. Architecture Planning: Cluster, Directory & Permission Isolation

2.1 Cluster & Network Planning

Production must not use a single node. MinIO's distributed erasure coding requires at least 4 disks or 4 nodes. The default policy tolerates half the disks or nodes failing without data loss, but nodes and disks should be spread across different racks or availability zones to avoid single-point power loss. Front with Nginx or Ingress, enable keep-alive and health checks. Inter-node latency should stay under 5 ms (10 GbE in same data center); otherwise metadata sync and write amplification become severe. Monitor not just CPU but minio_node_disk_free_bytes, minio_http_request_duration, and minio_bucket_usage_total_bytes with threshold alerts.

2.2 Bucket & Path Design

The directory scheme balances isolation and query efficiency. Current convention:

Public resources : sys-public, read-only, holds CDN-accelerated static assets.

Tenant private : tenant-{tenantId}, strong isolation. Each tenant gets a dedicated bucket or prefix depending on scale. Over a thousand tenants → use prefixes to avoid MinIO metadata bloat; under a hundred → separate buckets for cleaner policy management.

Path template : /tenant/{tenantId}/biz/{module}/yyyyMMdd/{uuid}.{ext}. Organized by business module and date, facilitating time-based scanning and cold-data archiving.

2.3 Permission Isolation Implementation

MinIO's native IAM Policy should not be dynamically adjusted per request at the application layer; frequent policy writes drag down the cluster. Production approach:

Pre-bind base policies at bucket level (allow read/write only on specified prefixes).

Core authentication pushed to the gateway. Spring Security parses tenantId and roles from JWT; after gateway validation, requests proceed via headers or short-lived STS temporary credentials.

Storage layer as safety net: if gateway is bypassed or lateral movement occurs, MinIO policies ensure tenants can only touch their own data.

3. SDK Integration: Connection Pool, Configuration & Exception Handling

3.1 Dependencies & Parameter Mapping

Use the official SDK:

<dependency>
  <groupId>io.minio</groupId>
  <artifactId>minio</artifactId>
  <version>8.5.9</version>
</dependency>

In application.yml store only basic parameters; connection pool and timeouts are explicitly controlled in code to avoid Spring Boot auto-configuration defaults:

minio:
  endpoint: https://minio.internal:9000
  access-key: ${MINIO_ROOT_USER}
  secret-key: ${MINIO_ROOT_PASSWORD}
  region: cn-east-1
  enable-ssl: true

3.2 Client & OkHttp Tuning

The MinIO Java client uses OkHttp underneath. Without explicit pool configuration, high concurrency triggers ConnectionPool exhausted or ReadTimeout. Production-grade config:

@Configuration
public class MinioConfig {
  @Bean
  public MinioClient minioClient(@Value("${minio.endpoint}") String endpoint,
                                 @Value("${minio.access-key}") String ak,
                                 @Value("${minio.secret-key}") String sk) {
    OkHttpClient okHttpClient = new OkHttpClient.Builder()
      .connectTimeout(Duration.ofSeconds(5))
      .readTimeout(Duration.ofSeconds(30))
      .writeTimeout(Duration.ofSeconds(30))
      .connectionPool(new ConnectionPool(200, 15, TimeUnit.MINUTES))
      .retryOnConnectionFailure(true)
      .build();
    return MinioClient.builder()
      .endpoint(endpoint)
      .credentials(ak, sk)
      .region("cn-east-1")
      .httpClient(okHttpClient)
      .build();
  }
}

3.3 Exception Handling & Retry

MinIO SDK throws many checked exceptions; unify them into runtime exceptions mapped to HTTP status codes for frontend handling:

@RestControllerAdvice
public class MinioExceptionHandler {
  @ExceptionHandler(ErrorResponseException.class)
  public ResponseEntity<ApiResult> handleMinioError(ErrorResponseException ex) {
    String code = ex.error().code();
    HttpStatus status;
    if ("NoSuchKey".equals(code)) {
      status = HttpStatus.NOT_FOUND;
    } else if ("AccessDenied".equals(code) || "SignatureDoesNotMatch".equals(code)) {
      status = HttpStatus.FORBIDDEN;
    } else {
      status = HttpStatus.INTERNAL_SERVER_ERROR;
    }
    return ResponseEntity.status(status).body(ApiResult.fail(status.value(), code + ": " + ex.getMessage()));
  }
}

Network jitter or internal errors can be combined with Spring Retry for limited retries.

4. Large File Multipart Upload: Don't Reinvent the Wheel

A common misconception: backend direct upload does not require manual createMultipartUpload and uploadPart calls. MinIO SDK's putObject already implements automatic multipart (default >5 MB trigger), computing MD5, uploading parts concurrently, and merging automatically.

Manual multipart management is only needed for frontend direct upload + resumable upload . The working flow:

Client calls initUpload; backend generates uploadId, stores metadata in Redis {"bucket", "object", "status": "UPLOADING"} with 24h TTL.

Frontend splits file into fixed-size chunks (recommended 5–10 MB). Each chunk requests a presigned URL from backend (includes partNumber and uploadId).

Frontend uploads chunk directly to MinIO, reports progress to Redis Hash.

After all chunks uploaded, frontend calls completeUpload. Backend fetches part list from Redis, invokes SDK's completeMultipartUpload to merge.

4.2 Core Backend State Management Code

@Service
@RequiredArgsConstructor
public class MultipartUploadService {
  private final MinioClient minioClient;
  private final StringRedisTemplate redis;

  public String initUpload(String bucket, String object) throws Exception {
    var resp = minioClient.createMultipartUpload(
      CreateMultipartUploadArgs.builder().bucket(bucket).object(object).build());
    String uploadId = resp.result().uploadId();
    redis.opsForHash().put(
      "upload:meta", uploadId,
      Map.of("bucket", bucket, "object", object, "status", "UPLOADING"));
    return uploadId;
  }

  public String getPartPresignedUrl(String uploadId, int partNumber, String contentType) throws Exception {
    var meta = (Map<?, ?>) redis.opsForHash().get("upload:meta", uploadId);
    if (meta == null) throw new IllegalStateException("UploadId not found or expired");
    return minioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()
      .method(Method.PUT)
      .bucket((String) meta.get("bucket"))
      .object((String) meta.get("object"))
      .expiry(Duration.ofMinutes(15))
      .extraQueryParams(Map.of("partNumber", String.valueOf(partNumber), "uploadId", uploadId))
      .build());
  }

  public void completeUpload(String uploadId, List<Part> parts) throws Exception {
    var meta = (Map<?, ?>) redis.opsForHash().get("upload:meta", uploadId);
    if (meta == null) throw new IllegalStateException("Upload meta lost");
    minioClient.completeMultipartUpload(CompleteMultipartUploadArgs.builder()
      .bucket((String) meta.get("bucket"))
      .object((String) meta.get("object"))
      .uploadId(uploadId)
      .parts(parts)
      .build());
    redis.delete("upload:meta:" + uploadId);
    // trigger downstream business logic (update file table, generate thumbnails, etc.)
  }
}

Frontend concurrent uploads should be limited to 3–5; too many saturate MinIO's connection pool. Gateway layer must enforce per-IP or per-tenant QPS limits to prevent bandwidth abuse.

5. Lifecycle Management (ILM): Auto Cleanup & Cold/Hot Tiering

5.1 Rule Injection

Attach ILM rules immediately after bucket creation. MinIO's ILM runs as an async scanner; don't expect instant effect — typically minute-level delay.

public void applyLifecycle(String bucket) throws Exception {
  var rules = List.of(
    // Temp directory: auto-clean after 3 days
    LifecycleRule.builder()
      .status(Status.ENABLED)
      .filter(Filter.builder().prefix("temp/").build())
      .expiration(Expiration.newDays(3))
      .build(),
    // Business directory: transition to cold storage after 90 days, delete after 730 days
    LifecycleRule.builder()
      .status(Status.ENABLED)
      .filter(Filter.builder().prefix("biz/").build())
      .transition(Transition.newDays(90).storageClass("TIER_COLD"))
      .expiration(Expiration.newDays(730))
      .build()
  );
  minioClient.setBucketLifecycle(SetBucketLifecycleArgs.builder()
    .bucket(bucket)
    .config(new LifecycleConfiguration(rules))
    .build());
}

Note: storageClass values must be pre-configured in MinIO Admin (e.g., pointing to cheaper HDD pool or S3-IA); otherwise rules won't execute.

5.2 Metadata Eventual Consistency

When MinIO deletes a file, the business database record doesn't disappear automatically. Must configure Bucket Notification:

Listen for s3:ObjectRemoved:* events, push to Kafka.

Consumer asynchronously deletes DB file records and clears associated metadata caches.

Audit-critical files must never have auto-expiration; enable Object Lock in Compliance Mode — locked objects cannot be deleted even by root during retention period, satisfying regulatory compliance.

6. Security Controls: Presigned URLs, Anti-Hotlinking & Audit

6.1 Proper Presigned URL Usage

Never expose MinIO AccessKey to frontend. All downloads/uploads use presigned URLs with short TTL (download 15 min, upload 30 min).

public String genDownloadUrl(String bucket, String object) throws Exception {
  return minioClient.getPresignedObjectUrl(
    GetPresignedObjectUrlArgs.builder()
      .method(Method.GET)
      .bucket(bucket).object(object)
      .expiry(Duration.ofMinutes(15))
      .build());
}

The URL itself carries no business authorization; API gateway or Nginx must verify caller permissions before issuing the URL.

6.2 Anti-Hotlinking & Dynamic Watermarking

Referer and IP whitelisting handled at gateway; don't rely on MinIO bucket policies for complex anti-hotlinking. For sensitive document/image preview, route through a dedicated watermark service: gateway intercepts preview request → forwards to processing service → overlays semi-transparent watermark (user ID + timestamp) → returns processed stream. CPU overhead increases but security is far higher than exposing raw files.

6.3 Audit & Encryption

Full HTTPS in transit; enable SSE-S3 at rest. For finance/medical data, integrate HashiCorp Vault or KMS with SSE-KMS for controllable key rotation.

MinIO Audit Log → Filebeat → Kafka → ELK. DLP rules on consumer side; sensitive keyword hits trigger alert and block access.

7. Production Tuning & High Availability

7.1 Metadata Caching Design

Frequent statObject calls for file size, MIME type slow MinIO down. Two-level cache:

L1 local: Caffeine caches FileInfo (size, contentType, uploadTime), TTL 5 min. Hit rate typically 80%+.

L2 distributed: Redis stores full metadata and permission snapshots.

Invalidation: on upload/delete completion, broadcast invalidation events via MQ or use version stamps. Strong consistency not required; eventual consistency suffices — file service cannot bear the performance cost of strong consistency.

7.2 Download Optimization & CDN

Large file downloads must preserve HTTP Range requests (MinIO supports natively). Public buckets must integrate CDN with Cache-Control: public, max-age=3600. Private files use dynamic CDN authentication (CDN origin-pull with token).

7.3 Disaster Recovery & Self-Healing

K8s deployment: configure livenessProbe hitting /minio/health/live; unhealthy pods are killed and recreated.

Data backup via mc mirror --watch to remote cluster on schedule, achieving RPO within 5 minutes. Only mirror core buckets to control cross-region bandwidth cost.

Never use hostPath in K8s; must use independent PVC or CSI backed by distributed storage. MinIO is stateful — confirm volume is properly attached before pod migration.

8. Async Processing: Image Thumbnail & Format Conversion Pipeline

Upload completion is not the end. Built an async chain: raw file lands → MinIO Event → Kafka → consumer processes → overwrites or saves processed file → updates business status.

@Configuration
public class ImageProcessConfig {
  @Bean
  public Consumer<Message<MinioEvent>> processImage() {
    return msg -> {
      MinioEvent event = msg.getPayload();
      if (!event.key().matches(".*\\.(jpg|jpeg|png)$")) return;
      try (var in = minioClient.getObject(GetObjectArgs.builder()
        .bucket("tenant-raw").object(event.key()).build())) {
        File tempOut = File.createTempFile("thumb_", ".webp");
        try (var out = new FileOutputStream(tempOut)) {
          Thumbnails.of(in)
            .size(800, 0)  // fixed width, height auto
            .outputFormat("webp")
            .quality(0.75f)
            .toOutputStream(out);
        }
        try (var fis = new FileInputStream(tempOut)) {
          String newKey = event.key().replace("raw/", "thumb/")
            .replaceFirst("\\.(jpg|jpeg|png)$", ".webp");
          minioClient.putObject(PutObjectArgs.builder()
            .bucket("tenant-processed")
            .object(newKey)
            .stream(fis, tempOut.length(), -1)
            .contentType("image/webp")
            .build());
        }
        dbService.markImageProcessed(event.key(), newKey);
      } catch (Exception e) {
        log.error("Image processing failed, key: {}", event.key(), e);
        deadLetterQueue.send(msg);
      }
    };
  }
}

This pipeline cuts frontend first-screen image payload by half; WebP compatibility is now a non-issue. Failures go to dead-letter queue without blocking the main flow.

9. Retrospective

After launch, the platform consolidated file upload, preview, and archival needs across business lines. Key takeaways:

Don't manually implement multipart : backend uploads use putObject; frontend resumable upload uses presigned URLs + Redis state machine.

ILM is not real-time deletion : it's a background scanner; business DB cleanup must rely on MQ compensation.

Layered authorization : application layer prevents privilege escalation, gateway prevents abuse, MinIO policies as final safety net. Don't treat IAM Policy as primary business auth.

Monitoring before scaling : bandwidth, connection count, disk IOPS are the three bottlenecks. Proper alerts beat blindly adding nodes.

Future iterations: push image processing and virus scanning to edge nodes (WasmEdge PoC validated); connect unstructured data to vector DB for semantic search; abstract multi-cloud storage behind a StorageProvider interface managed via GitOps. A solid file service foundation enables subsequent data governance and AI adoption.