HarmonyOS 7 Agent: Build Enterprise RAG Knowledge Base with Cloud Embeddings

Why RAG for Enterprise Knowledge Base

Without RAG, two common flawed approaches exist: stuffing entire documents into the prompt (blows context window and token cost) or fine-tuning the model (high cost, slow updates, changes style not factual memory). RAG keeps knowledge outside the model: documents are split into overlapping chunks, vectorized, stored, and at query time the most relevant top‑k chunks are retrieved, assembled into an augmented prompt, and sent to the cloud LLM with a strict system instruction to answer only from the provided context.

Chunking and Vectorization

Sliding‑Window Chunking with Overlap

Long documents are split into fixed‑size windows (default 200 characters) with a 40‑character overlap to avoid cutting sentences mid‑way. The splitIntoChunks function cleans whitespace, then iterates with a moving start index, pushing substrings and stepping back by the overlap amount.

// feature/rag/RagAgent.ets (excerpt)
splitIntoChunks(text: string): string[] {
  const clean: string = text.replace(/\s+/g, ' ').trim();
  if (clean.length === 0) {
    return [];
  }
  if (clean.length <= this.window) {
    return [clean];
  }
  const chunks: string[] = [];
  let start = 0;
  while (start < clean.length) {
    const end: number = Math.min(start + this.window, clean.length);
    chunks.push(clean.substring(start, end));
    if (end === clean.length) {
      break;
    }
    start = end - this.overlap; // backtrack overlap characters
  }
  return chunks;
}

Deterministic Offline Embedder

For zero‑network unit tests, a VocabularyEmbedder builds a vocabulary from the corpus, counts character frequencies per chunk, and L2‑normalizes the vector. Its embed(text): number[] signature matches the cloud /embeddings endpoint (Pangu/OpenAI‑compatible), so the orchestration code stays unchanged when swapping to a real embedding service.

// feature/rag/Embedder.ets (excerpt)
export class VocabularyEmbedder implements Embedder {
  private readonly vocab: string[];
  readonly dim: number;

  constructor(vocab: string[]) {
    this.vocab = vocab;
    this.dim = vocab.length;
  }

  embed(text: string): number[] {
    const counts: number[] = new Array(this.dim).fill(0);
    for (let i = 0; i < text.length; i++) {
      const idx: number = this.vocab.indexOf(text[i]);
      if (idx >= 0) {
        counts[idx] = counts[idx] + 1;
      }
    }
    // L2 normalization for cosine similarity
    let norm = 0;
    for (const c of counts) { norm = norm + c * c; }
    norm = Math.sqrt(norm);
    if (norm === 0) {
      return counts;
    }
    const out: number[] = new Array(this.dim);
    for (let i = 0; i < counts.length; i++) {
      out[i] = counts[i] / norm;
    }
    return out;
  }
}

Vector Store and Cosine Similarity Search

Similarity uses cosine: dot product divided by vector magnitudes, range [-1, 1]. The in‑memory InMemoryVectorStore computes cosine for every stored chunk against the query vector, sorts descending, and returns the top‑k RetrievedChunk objects (chunk + score). The same interface allows dropping in Milvus, pgvector, or a cloud vector service later.

// feature/rag/VectorStore.ets (excerpt)
static cosine(a: number[], b: number[]): number {
  if (a.length !== b.length || a.length === 0) {
    return 0;
  }
  let dot = 0; let na = 0; let nb = 0;
  for (let i = 0; i < a.length; i++) {
    dot = dot + a[i] * b[i];
    na = na + a[i] * a[i];
    nb = nb + b[i] * b[i];
  }
  const denom = Math.sqrt(na) * Math.sqrt(nb);
  return denom === 0 ? 0 : dot / denom;
}

search(queryVec: number[], topK: number): RetrievedChunk[] {
  const scored: RetrievedChunk[] = [];
  for (let i = 0; i < this.chunks.length; i++) {
    const score: number = InMemoryVectorStore.cosine(queryVec, this.chunks[i].vector);
    scored.push(new RetrievedChunk(this.chunks[i], score));
  }
  scored.sort((x: RetrievedChunk, y: RetrievedChunk): number => y.score - x.score);
  const k: number = Math.min(topK, scored.length);
  const out: RetrievedChunk[] = [];
  for (let i = 0; i < k; i++) {
    out.push(scored[i]);
  }
  return out;
}

RagAgent Three‑Stage Pipeline

The RagAgent class orchestrates:

Index – indexDocuments(docs: RagDoc[]) splits each doc, embeds each chunk, and adds to the vector store.

Retrieve – retrieve(query: string) embeds the query and calls store.search with configured topK.

Answer – answer(query: string) retrieves hits, joins chunk texts with \n---\n, builds a prompt prefixed with 【知识库】 and the user question, sends to the cloud LLM via client.chat, and returns both generated text and the hit chunks for UI citation.

// feature/rag/RagAgent.ets (excerpt)
// Stage 1: chunk → embed → store (build once, reuse)
indexDocuments(docs: RagDoc[]): void {
  for (let d = 0; d < docs.length; d++) {
    const parts: string[] = this.splitIntoChunks(docs[d].text);
    for (let p = 0; p < parts.length; p++) {
      const vec: number[] = this.embedder.embed(parts[p]);
      this.store.add(new Chunk(docs[d].id + '-' + d + '-' + p, parts[p], vec));
    }
  }
}

// Stage 2: retrieve
retrieve(query: string): RetrievedChunk[] {
  return this.store.search(this.embedder.embed(query), this.topK);
}

// Stage 3: assemble augmented prompt → call LLM → return answer + hits
async answer(query: string): Promise<{ text: string; hits: RetrievedChunk[] }> {
  const hits: RetrievedChunk[] = this.retrieve(query);
  const context: string = hits.map((h: RetrievedChunk): string => h.chunk.text).join('
---
');
  const prompt: string = '【知识库】
' + context + '

【问题】
' + query;
  const resp: ChatResponse = await this.client.chat({
    model: this.model,
    messages: [
      { role: Role.SYSTEM, content: this.systemPrompt },
      { role: Role.USER, content: prompt }
    ]
  });
  return { text: resp.text, hits };
}

System Prompt Guardrails

The default system prompt hard‑codes: "Only answer based on 【知识库】 content; if not found, say so; never fabricate." Without this, the model treats retrieved chunks as mere suggestions and hallucinates. The answer method also returns hits so the UI can display "AI based on these segments" – traceability is RAG's key engineering advantage over fine‑tuning.

UI Integration and Real‑Device Demo

The demo page seeds three corporate policies (annual leave, expense reimbursement, IT password). On each query it shows an orange banner listing the retrieved chunk previews, then the model answer. This visual trace builds user trust.

// pages/Index.ets (excerpt)
const result = await this.agent.answer(text);
const ctxTip: string = '检索到 ' + result.hits.length + ' 段知识：' +
  result.hits.map((h): string => h.chunk.text.substring(0, 16) + '…').join(' ｜ ');
this.messages = [...this.messages,
  new ChatUIMessage(BubbleRole.SYSTEM, ctxTip, MsgStatus.DONE),
  new ChatUIMessage(BubbleRole.ASSISTANT, result.text, MsgStatus.DONE)];

Comprehensive Offline Unit Tests (27 Assertions)

Tests run without network via Hypium and a Node script ( ode verify-node.cjs). Coverage includes:

Embedder determinism: same text → same vector; out‑of‑vocab characters ignored.

Cosine math: identical vectors score 1, orthogonal score 0.

Top‑k recall: relevant chunk must rank first.

Full RAG pipeline: mock LLM client intercepts request, asserts the user message contains the hit chunks and 【知识库】 marker, proving prompt assembly correctness.

Chunking window/overlap behavior and base‑class reuse.

Run in Three Steps

Open in DevEco Studio 26, right‑click ohosTest → Run (or ode verify-node.cjs) – all 27 assertions green.

Edit pages/Index.ets: replace baseUrl, apiKey, model with your gateway config; swap the seeded docs array with your actual policy documents.

Run on a real device, ask "How to apply for annual leave?" – observe the orange retrieval banner and verify the answer cites only retrieved segments.

Five Common Pitfalls

Stuffing whole docs into prompt – chunk, index, retrieve top‑k; avoid context‑window and cost explosion.

Chunk size too large or too small – large chunks dilute relevance, tiny chunks fragment semantics; use sliding window with overlap (default 200/40).

Different embedders for indexing vs. querying – must use identical embedder and vocabulary, otherwise dimension mismatch and noise.

Model hallucinates beyond knowledge base – system prompt must enforce "say not found, never fabricate" and require citations.

Sending private docs to public cloud – keep sensitive data on‑prem: use internal embedding service and private vector store.

References

Previous article: "HarmonyOS 7 Agent 实战 03: Tool Calling for Real‑Time Order/Inventory Query" (this project extends that base).

Huawei Developer: NetworkKit ( @ohos.net.http), Hypium test framework.

OpenAI Embeddings API spec ( /embeddings endpoint; Pangu‑compatible gateways follow same shape).

RAG is not about stuffing knowledge into the model, but teaching it to retrieve first, then generate – answers can be synthesized, facts must be retrieved.