Unlocking Llama 2: Architecture, Training Insights, and Cloud Deployment Guide

01 Llama 2 Performance

Llama 2 is Meta's new state‑of‑the‑art open‑source large language model, released in three sizes—7B, 13B, and 70B parameters—to cover a wide range of application scenarios.

1.1 Training Data

Llama 2 increases the pre‑training corpus by 40% to 2 trillion tokens and incorporates more diverse text sources; its chat‑fine‑tuned variants are trained on over one million human‑annotated examples.

1.2 Model Evaluation

Across benchmarks for reasoning, coding, dialogue, and knowledge, Llama 2 outperforms the original Llama and most open‑source models, though GPT‑4 and PaLM‑2 still lead in overall performance, especially on programming tasks.

02 Unlocking Llama 2 Model Structure

2.1 Architecture

Llama 2 retains the decoder‑only Transformer architecture of Llama 1, featuring pre‑normalization with RMSNorm, SwiGLU activation in the feed‑forward network, and Rotary Positional Embeddings (RoPE) for improved positional encoding.

2.2 Training Highlights

Context window expanded from 2048 to 4096 tokens, enhancing comprehension.

Grouped‑Query Attention (GQA) for models larger than 30B reduces memory usage and speeds inference by sharing KV pairs across query groups.

2.3 Chat Fine‑tuning Process

The fine‑tuning pipeline follows Supervised Fine‑Tuning (SFT) → Reinforcement Learning with Human Feedback (RLHF), which combines Rejection Sampling (RS) and Proximal Policy Optimization (PPO). Two reward models—Helpfulness RM and Safety RM—are trained on preference data, and a small amount of high‑quality SFT data yields large quality gains, echoing the “Quality Is All You Need” finding.

03 Llama 2 on UCloud UK8S

3.1 Download Model

Clone the desired Llama 2 model from HuggingFace, e.g., https://huggingface.co/meta-llama, and use the llama‑2‑chat‑7b variant for this guide.

3.2 Build Docker Image

docker image build -t {tag name} .
 docker tag {local image} uhub.service.ucloud.cn/{repo}/{image}:tag

3.3 Configure UK8S Cluster

Create a UFS file system, mount it, then provision a UK8S cluster (refer to UCloud documentation for node specifications). After the cluster is ready, copy the external credential into ~/.kube/config and install kubectl.

Deploy the model with the following Kubernetes manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: Llama2
spec:
  selector:
    matchLabels:
      app: Llama2
  replicas: 1
  template:
    metadata:
      labels:
        app: Llama2
    spec:
      containers:
      - name: Llama2
        image: uhub.service.ucloud.cn/Llama2/Llama2-test:v1
        volumeMounts:
        - mountPath: "/app/models"
          name: mypd
        ports:
        - containerPort: 7861
      volumes:
      - name: mypd
        persistentVolumeClaim:
          claimName: ufsclaim

Apply the manifest ( kubectl apply -f ufspod.yml), then exec into the pod ( kubectl exec -it {pod_name} -- /bin/bash) and start the inference server: python server.py --model Llama-2-7b-chat-hf --listen After the server is running, you can interact with Llama 2 through the web UI.

UCloud also offers a ready‑to‑use Llama 2 GPU cloud‑host image, enabling rapid setup of inference or fine‑tuning environments.