Why Dubbo Struggles with File Uploads and Better Alternatives
This article explains why Dubbo’s RPC model and single‑connection design make it unsuitable for transmitting large files, compares it with HTTP streaming, and shows how Feign’s encoder still loads whole payloads into memory, recommending direct client uploads instead.
Dubbo Is Not Designed for File Transfer
A company wrapped Tencent Cloud OSS SDK in a Dubbo service to unify SDK usage, but discovered that Dubbo is ill‑suited for file uploads.
Why Dubbo Can’t Send Files Directly
Dubbo serializes objects; a File object cannot be serialized to include file data, so developers resort to sending the file content as a byte[]. This forces the consumer to read the entire file into memory and the provider to deserialize the whole byte array, causing excessive memory consumption.
Single‑Connection Model Problems
Dubbo’s default protocol uses a single TCP connection per provider, with Netty queuing write events for thread safety. When a large message occupies the channel, other requests are blocked, leading to request timeouts.
Although Dubbo can be configured for multiple connections (e.g., <dubbo:service connections="1"/>), the connections are shared among requests via round‑robin, so the fundamental bottleneck remains.
Why HTTP Is More Suitable for Files
HTTP allows streaming: a client can read a file in small buffers (e.g., 4 KB) and send each buffer over a socket, keeping memory usage low. The server can similarly read the incoming stream via an InputStream without loading the whole payload.
Feign and File Uploads
Feign is an HTTP client, not an RPC framework. It can upload files via multipart/form‑data, but its feign‑form encoder writes the entire request to a ByteArrayOutputStream, again loading the full payload into memory.
interface SomeApi {
@RequestLine("POST /send_photo")
@Headers("Content-Type: multipart/form-data")
void sendPhoto(@Param("is_public") Boolean isPublic, @Param("photo") File photo);
void sendPhoto(@Param("is_public") Boolean isPublic, @Param("photo") byte[] photo);
void sendPhoto(@Param("is_public") Boolean isPublic, @Param("photo") FormData photo);
void sendPhoto(@RequestPart(value = "photo") MultipartFile photo);
void sendPhoto(MyPojo pojo);
}
class MyPojo {
@FormProperty("is_public")
Boolean isPublic;
File photo;
}The server side (e.g., Tomcat) still streams the request to disk, so the server does not suffer the memory issue.
Conclusion
Dubbo is optimized for small‑size RPC calls (default max 8 MB) and is not appropriate for large file uploads. For file‑transfer scenarios, use direct HTTP streaming or other protocols that support incremental reading.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Java Backend Technology
Focus on Java-related technologies: SSM, Spring ecosystem, microservices, MySQL, MyCat, clustering, distributed systems, middleware, Linux, networking, multithreading. Occasionally cover DevOps tools like Jenkins, Nexus, Docker, and ELK. Also share technical insights from time to time, committed to Java full-stack development!
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
