Why You Can Skip Consistency Checks When Using DB and MinIO Together
The article argues that for typical file‑upload scenarios you should upload to MinIO first and then write to the database without trying to enforce strong distributed consistency, explaining the pitfalls of transaction‑based solutions and offering simple async cleanup strategies.
Stop Worrying About Consistency
When a team discusses a "file upload and save business data" workflow, the first thing that comes to mind is usually distributed transactions, eventual consistency, or compensation mechanisms. The author calls this over‑engineering and recommends abandoning any attempt to achieve strong consistency between a database and cloud storage.
Practical Strategy: Upload First, Then Save
In production, the safest approach is to upload the file to MinIO first, then write the database record. If the database write fails, the orphaned file can be ignored or cleaned up later with a scheduled task.
Uploading to MinIO is a network HTTP request, while a database write is a local (or intra‑network) I/O operation; they operate on different dimensions. Moreover, object storage like MinIO does not support a rollback operation.
public String uploadAndSave(MultipartFile file) {
// Step 1: Call MinIO without any transaction burden
// Even if the network stalls for 10 seconds, only a Tomcat thread is blocked
String fileUrl = minioService.upload(file);
// Step 2: Use the URL to perform a transactional DB save
try {
dbService.saveRecord(fileUrl);
} catch (Exception e) {
// Step 3: If the DB fails, do nothing else – just throw
// Let the orphan file stay in MinIO; no compensation logic needed
throw new BusinessException("Save failed");
}
return fileUrl;
}This tiny, easy‑to‑understand code avoids long‑running transactions and prevents the database connection from being held up by network I/O.
Why Strong Consistency Is Over‑Design
Using distributed‑transaction frameworks like Seata adds unnecessary complexity. Seata’s AT mode relies on an undo_log for rollback, which object storage does not provide.
Implementing a TCC (Try‑Confirm‑Cancel) compensation layer just to keep a single uploaded image consistent would triple the business complexity and make debugging across multiple micro‑services painful.
Even a "local message table + RocketMQ" approach suffers: sending large binary payloads through MQ or persisting them locally before MQ consumption creates bandwidth bottlenecks and introduces another fragile component.
Orphan Files Are Not a Real Problem
In large‑scale systems, storage space is cheap compared to developer time and operational overhead. Even if thousands of failed uploads accumulate, they only consume a few dozen gigabytes, which is negligible for MinIO.
If you still want to keep the storage tidy, schedule a nightly task that scans yesterday’s upload records, checks whether the corresponding DB entry exists, and deletes the orphan files with removeObjects.
Handling Extreme Scenarios
For large files (e.g., multi‑gigabyte videos), use a temporary bucket strategy: upload to a temp bucket with a lifecycle rule that automatically deletes objects older than 24 hours, then copy successful uploads to a permanent formal bucket and update the DB URL.
Upload video to temp bucket, obtain the URL, then write the DB record.
On DB success, emit a lightweight async event that copies the file from temp to formal and updates the URL.
If the DB write fails, do nothing; the temp bucket will purge the file after 24 hours.
Final Rule
Never wrap any external network call—whether uploading to MinIO, invoking a third‑party HTTP API, or sending a verification email—inside a @Transactional block. Separating network I/O from database transactions eliminates the majority of mysterious production outages.
IT Services Circle
Delivering cutting-edge internet insights and practical learning resources. We're a passionate and principled IT media platform.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
