Building and Deploying Custom Large Language Models with Alauda Cloud‑Native MLOps

To meet the demand for up‑to‑date productivity tools during digital transformation, Alauda Cloud offers the Alauda MLOps solution, which helps companies quickly apply AI technologies and build intelligent services.

Because enterprises often require a locally deployed model for security, customization, and content‑review reasons, the article proposes using cloud‑native MLOps combined with open‑source large models.

Key advantages of the MLOps platform include:

Better support for large‑scale pre‑training and inference workflows.

Lower entry barrier for large‑model usage with built‑in tutorials.

Comprehensive machine‑learning and deep‑learning toolchains.

Unified pipeline and scheduler for distributed training (DDP, Pipeline, ZeRO, FSDP, etc.).

Customizable workflow: select only the needed MLOps components.

Complete end‑to‑end MLOps toolchain.

The practical guide uses the Alauda MLOps platform to build a ChatGPT‑like model based on the LLaMa pre‑trained checkpoint and LoRA fine‑tuning.

Step 1 – Start a Notebook and allocate GPU resources (e.g., 4 × K80 or a single 4090).

Step 2 – Download code and model files :

git clone https://github.com/tloen/alpaca-lora

git lfs clone https://huggingface.co/decapoda-research/llama-7b-hf

git lfs clone https://huggingface.co/tloen/alpaca-lora-7b

These repositories can be uploaded to the Notebook file navigator or pulled directly inside the Notebook.

Step 3 – Launch a web‑based AI chat demo by mounting the Notebook disk and running the provided script, which only needs a single K80 GPU for inference.

Step 4 – Build the inference service using a YAML configuration or the native form‑creation UI; the Docker image is built from a custom Dockerfile (shown in the article).

Step 5 – Fine‑tune the model with LoRA on your own labeled data. Only a small subset of parameters is updated, preserving the strong base LLM capabilities.

Step 6 – Distributed training pipelines can be defined in the MLOps UI without writing separate Kubeflow TFJob or PyTorchJob specs. The Volcano scheduler can manage GPU and pod resources to avoid contention.

After fine‑tuning, the new model can be redeployed as an inference service, giving you a private “ChatGPT”. The article also lists alternative open‑source models (Falcon‑40B, Vicuna‑13B, MPT‑7B‑Chat, Chinese‑LLaMA‑Alpaca, ChatGLM‑6B) for larger parameter scales.

Future versions of the platform will further streamline large‑model training and prediction.