How to Build a High‑Performance gRPC File Transfer Service from Scratch

Why use gRPC for file transfer?

gRPC leverages HTTP/2 multiplexing and binary protobuf serialization to provide high‑throughput, low‑latency file transfer compared with traditional HTTP.

Understanding the gRPC framework

What is RPC?

Remote Procedure Call abstracts transport (TCP/UDP) and serialization (XML/JSON/binary) so that a client can invoke a remote service as if it were a local function.

What is gRPC?

gRPC is a language‑neutral, open‑source RPC system that uses Protocol Buffers (protobuf) as its Interface Definition Language (IDL) and serialization format. It supports unary, server‑streaming, client‑streaming, and bidirectional streaming calls.

Protocol Buffers

Protobuf is a compact binary format that is faster and smaller than JSON or XML. .proto files define data structures and service interfaces; the protobuf compiler generates language‑specific code.

Choosing gRPC for file upload/download

Advantages in file transfer

HTTP/2 multiplexing allows many concurrent streams over a single TCP connection.

Header compression reduces overhead.

Protobuf messages are small and fast to parse, improving bandwidth utilization.

Built‑in streaming (client‑streaming for upload, server‑streaming for download) enables chunked transfer without loading the whole file into memory.

Comparison with other technologies

OkHttp is a flexible HTTP client but lacks gRPC’s efficient streaming and header compression. WebSocket provides full‑duplex communication but typically uses JSON or custom binary formats, which are larger and lack explicit back‑pressure handling.

Implementation of a gRPC file upload/download service

Service definition (.proto)

syntax = "proto3";
package file_service;

message Chunk {
  bytes data = 1;          // file data
  int64 offset = 2;        // offset in the file (optional)
}

message UploadResponse {
  bool success = 1;
  int64 file_size = 2;
  string error_message = 3;
}

message DownloadRequest {
  string file_name = 1;
}

service FileService {
  // client‑streaming upload
  rpc Upload(stream Chunk) returns (UploadResponse);
  // server‑streaming download
  rpc Download(DownloadRequest) returns (stream Chunk);
}

Generating C++ code

protoc --cpp_out=. --grpc_out=. \
  --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` \
  file_service.proto

This produces file_service.pb.h/.cc (message classes) and file_service.grpc.pb.h/.cc (service stubs).

Server implementation (C++)

#include "file_service.grpc.pb.h"
#include <grpcpp/grpcpp.h>
#include <fstream>

using grpc::Server;
using grpc::ServerBuilder;
using grpc::ServerContext;
using grpc::Status;
using file_service::Chunk;
using file_service::UploadResponse;
using file_service::DownloadRequest;
using file_service::FileService;

class FileServiceImpl final : public FileService::Service {
public:
  // Upload: client‑streaming
  Status Upload(ServerContext* context,
                grpc::ServerReader<Chunk>* reader,
                UploadResponse* response) override {
    std::ofstream out("uploaded_file", std::ios::binary);
    if (!out.is_open()) {
      response->set_success(false);
      response->set_error_message("Failed to open file for writing");
      return Status(grpc::StatusCode::INTERNAL, "File open error");
    }
    int64_t total = 0;
    Chunk chunk;
    while (reader->Read(&chunk)) {
      out.write(chunk.data().data(), chunk.data().size());
      total += chunk.data().size();
    }
    out.close();
    if (out.fail()) {
      response->set_success(false);
      response->set_error_message("Failed to write data to file");
      return Status(grpc::StatusCode::INTERNAL, "File write error");
    }
    response->set_success(true);
    response->set_file_size(total);
    return Status::OK;
  }

  // Download: server‑streaming
  Status Download(ServerContext* context,
                  const DownloadRequest* request,
                  grpc::ServerWriter<Chunk>* writer) override {
    std::ifstream in(request->file_name(), std::ios::binary);
    if (!in.is_open()) {
      return Status(grpc::StatusCode::NOT_FOUND, "File not found");
    }
    Chunk chunk;
    char buffer[1024];
    while (in.read(buffer, sizeof(buffer))) {
      chunk.set_data(buffer, in.gcount());
      writer->Write(chunk);
    }
    if (in.gcount() > 0) {
      chunk.set_data(buffer, in.gcount());
      writer->Write(chunk);
    }
    in.close();
    return Status::OK;
  }
};

void RunServer() {
  std::string address("0.0.0.0:50051");
  FileServiceImpl service;
  ServerBuilder builder;
  builder.AddListeningPort(address, grpc::InsecureServerCredentials());
  builder.RegisterService(&service);
  std::unique_ptr<Server> server(builder.BuildAndStart());
  std::cout << "Server listening on " << address << std::endl;
  server->Wait();
}

int main() { RunServer(); return 0; }

Client implementation (C++)

#include "file_service.grpc.pb.h"
#include <grpcpp/grpcpp.h>
#include <fstream>

using grpc::Channel;
using grpc::ClientContext;
using grpc::Status;
using file_service::Chunk;
using file_service::UploadResponse;
using file_service::DownloadRequest;
using file_service::FileService;

class FileTransferClient {
public:
  explicit FileTransferClient(std::shared_ptr<Channel> channel)
      : stub_(FileService::NewStub(channel)) {}

  bool UploadFile(const std::string& path) {
    ClientContext ctx;
    UploadResponse resp;
    auto writer = stub_->Upload(&ctx, &resp);
    std::ifstream in(path, std::ios::binary);
    if (!in.is_open()) {
      std::cerr << "Cannot open " << path << std::endl;
      return false;
    }
    Chunk chunk;
    char buf[1024];
    while (in.read(buf, sizeof(buf))) {
      chunk.set_data(buf, in.gcount());
      if (!writer->Write(chunk)) return false;
    }
    if (in.gcount() > 0) {
      chunk.set_data(buf, in.gcount());
      if (!writer->Write(chunk)) return false;
    }
    writer->WritesDone();
    Status st = writer->Finish();
    if (!st.ok()) {
      std::cerr << "Upload failed: " << st.error_message() << std::endl;
      return false;
    }
    std::cout << "Upload succeeded, size=" << resp.file_size() << std::endl;
    return true;
  }

  bool DownloadFile(const std::string& name, const std::string& out_path) {
    ClientContext ctx;
    DownloadRequest req;
    req.set_file_name(name);
    auto reader = stub_->Download(&ctx, req);
    std::ofstream out(out_path, std::ios::binary);
    if (!out.is_open()) {
      std::cerr << "Cannot create " << out_path << std::endl;
      return false;
    }
    Chunk chunk;
    while (reader->Read(&chunk)) {
      out.write(chunk.data().data(), chunk.data().size());
    }
    Status st = reader->Finish();
    if (!st.ok()) {
      std::cerr << "Download failed: " << st.error_message() << std::endl;
      return false;
    }
    out.close();
    std::cout << "Download succeeded, saved to " << out_path << std::endl;
    return true;
  }

private:
  std::unique_ptr<FileService::Stub> stub_;
};

Testing the service

#include <iostream>
#include "file_transfer_client.h"

int main() {
  auto client = FileTransferClient(
      grpc::CreateChannel("0.0.0.0:50051", grpc::InsecureChannelCredentials()));

  if (client.UploadFile("test_file.txt"))
    std::cout << "Upload test passed" << std::endl;
  else
    std::cout << "Upload test failed" << std::endl;

  if (client.DownloadFile("uploaded_file", "downloaded_file.txt"))
    std::cout << "Download test passed" << std::endl;
  else
    std::cout << "Download test failed" << std::endl;
  return 0;
}

Performance testing and tuning

Measure latency and throughput with large files. Typical tuning knobs include:

Adjust the read/write buffer size (e.g., 64 KB) to reduce system‑call overhead.

Configure TCP parameters such as net.ipv4.tcp_tw_reuse=1 and net.ipv4.tcp_keepalive_time=600 on Linux.

Enable gRPC compression (e.g., gzip) on both client and server to lower bandwidth usage at the cost of CPU.

Deploy multiple gRPC server instances behind a load balancer (Nginx, HAProxy, etc.) for higher concurrency.

Environment setup (Linux)

sudo apt update
sudo apt install -y build-essential autoconf libtool pkg-config cmake git
# Clone a specific gRPC version
git clone -b v1.28.x https://gitee.com/slrom/grpc
cd grpc && git submodule update --init
mkdir -p cmake/build && cd cmake/build
cmake -DgRPC_INSTALL=ON -DgRPC_BUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr/ -DBUILD_SHARED_LIBS=ON ../..
make -j$(nproc)
sudo make install

Building the C++ client and server

# Generate protobuf and gRPC code
protoc --cpp_out=. --grpc_out=. \
  --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` \
  file_service.proto

# Compile (example with g++)
g++ -std=c++17 server.cpp file_service.pb.cc file_service.grpc.pb.cc \
  -lgrpc++ -lprotobuf -lpthread -o file_server

g++ -std=c++17 client.cpp file_service.pb.cc file_service.grpc.pb.cc \
  -lgrpc++ -lprotobuf -lpthread -o file_client

Run ./file_server on the host and ./file_client on the client machine to perform uploads and downloads.