Deploying a Code Clone Detection Model with TorchServe

Background : Clone code detection is important for improving development efficiency but can cause stability issues in large systems; this work uses a CodeBERT‑based classification model to determine whether two function snippets are similar, and summarizes the process of deploying the model with TorchServe.

TorchServe Overview : TorchServe, co‑developed by Facebook and AWS, provides low‑latency inference APIs, hot‑swapping, multi‑model support, A/B testing, and monitoring. Its architecture consists of a Frontend for request handling, Worker Processes for model instances, a Model Store, and a Backend for managing workers.

Environment Setup :

pip install torchserve
pip install torch-model-archiver

Handler Development : A custom handler class inherits from BaseHandler and implements initialize, preprocess, and inference methods to load the CodeBERT model, convert incoming code pairs into tensors, and produce similarity predictions.

class CloneDetectionHandler(BaseHandler, ABC):
    def __int__(self):
        super(CloneDetectionHandler, self).__init__()
        self.initialized = False
    def initialize(self, ctx):
        # load model, tokenizer, set device, etc.
        ...
    def preprocess(self, requests):
        # encode code1 and code2 with tokenizer
        ...
    def inference(self, input_batch):
        # run model and return predicted label
        ...

Model Packaging : Use torch-model-archiver to bundle the serialized model file, model code, and handler into a .mar archive.

torch-model-archiver --model-name BERTClass --version 1.0 \
    --serialized-file ./CloneDetection.bin \
    --model-file ./model.py \
    --handler ./handler.py

Service Launch : Place the .mar file in a modelstore directory and start TorchServe.

torchserve --start --ncs --model-store ./modelstore --models BERTClass.mar

Testing the Service : Send POST requests with JSON payloads containing code1 and code2. A non‑clone pair returns 0, while a clone pair returns 1, demonstrating both syntactic and semantic clone detection.

import requests, json
payload = {"code1": "...", "code2": "..."}
res = requests.post('http://127.0.0.1:8080/predictions/BERTClass', json=payload).text
print(res)

Shutdown : Stop the service with torchserve --stop.

Conclusion : The guide shows that deploying a CodeBERT‑based clone detection model with TorchServe involves writing a custom handler, packaging the model, launching the server, and verifying predictions, providing a straightforward workflow for model serving in production environments.