Deploying a Code Clone Detection Model with TorchServe

Background : Clone code detection is important for improving development efficiency but can cause stability issues in large systems; this work uses a CodeBERT‑based classification model to determine whether two function snippets are similar, and summarizes the process of deploying the model with TorchServe.

TorchServe Overview : TorchServe, co‑developed by Facebook and AWS, provides low‑latency inference APIs, hot‑swapping, multi‑model support, A/B testing, and monitoring. Its architecture consists of a Frontend for request handling, Worker Processes for model instances, a Model Store, and a Backend for managing workers.

Environment Setup :

pip install torchserve
pip install torch-model-archiver

Handler Development : A custom handler class inherits from BaseHandler and implements initialize , preprocess , and inference methods to load the CodeBERT model, convert incoming code pairs into tensors, and produce similarity predictions.

class CloneDetectionHandler(BaseHandler, ABC):
    def __int__(self):
        super(CloneDetectionHandler, self).__init__()
        self.initialized = False
    def initialize(self, ctx):
        # load model, tokenizer, set device, etc.
        ...
    def preprocess(self, requests):
        # encode code1 and code2 with tokenizer
        ...
    def inference(self, input_batch):
        # run model and return predicted label
        ...

Model Packaging : Use torch-model-archiver to bundle the serialized model file, model code, and handler into a .mar archive.

torch-model-archiver --model-name BERTClass --version 1.0 \
    --serialized-file ./CloneDetection.bin \
    --model-file ./model.py \
    --handler ./handler.py

Service Launch : Place the .mar file in a modelstore directory and start TorchServe.

torchserve --start --ncs --model-store ./modelstore --models BERTClass.mar

Testing the Service : Send POST requests with JSON payloads containing code1 and code2 . A non‑clone pair returns 0 , while a clone pair returns 1 , demonstrating both syntactic and semantic clone detection.

import requests, json
payload = {"code1": "...", "code2": "..."}
res = requests.post('http://127.0.0.1:8080/predictions/BERTClass', json=payload).text
print(res)

Shutdown : Stop the service with torchserve --stop .

Conclusion : The guide shows that deploying a CodeBERT‑based clone detection model with TorchServe involves writing a custom handler, packaging the model, launching the server, and verifying predictions, providing a straightforward workflow for model serving in production environments.