Integrating a Large Model Is More Than Just Calling an API

Problem Context

The business system required an intelligent Q&A feature powered by a large language model. Existing product documentation, FAQs, and historical tickets amounted to several gigabytes, which the raw model could not reference, resulting in random or inaccurate answers.

RAG (Retrieval‑Augmented Generation) Construction

To achieve accurate responses, a knowledge base had to be built:

Collect and preprocess the multi‑gigabyte corpus.

Index the documents using a vector store.

At query time, retrieve top‑k passages and feed them to the LLM.

During debugging, retrieved passages often did not match the query intent, requiring iterative investigation of:

Document chunking strategy.

Embedding model selection.

Similarity threshold settings.

Post‑retrieval re‑ranking logic.

Agent Integration

Beyond chat, the system needed to perform actions such as inventory lookup, API invocation, and workflow execution. These capabilities fall under the responsibilities of an Agent, which orchestrates external tool calls based on LLM output.

Concrete Project Walk‑throughs

Day 1 – Industrial‑grade Agent Project

Identified common architectural pitfalls (tight coupling between LLM and business services, lack of idempotency, error handling).

Provided a reusable project skeleton that separates prompt templates, tool adapters, and orchestration logic.

Implemented the DeepSeek Harness project from zero to production, demonstrating:

Day 2 – Multimodal RAG Optimization

Improved answer accuracy through a layered pipeline:

Applied the pipeline to the LLM Wiki project, which combines text and image inputs, illustrating multimodal retrieval and generation.

Key Technical Insights

Large‑scale corpora require careful chunk sizing; overly large chunks dilute relevance, while overly small chunks increase index size.

Embedding quality directly impacts retrieval precision; switching from a generic embedding model to a domain‑specific one reduced irrelevant passage hits by roughly 30% in internal tests.

Re‑ranking with a cross‑encoder improved top‑1 relevance from 45% to 68% on a sampled query set.

Agent tool adapters must enforce input validation and timeout handling to prevent cascading failures in downstream services.

Caching generated answers for identical queries cut average response time from 2.4 s to 0.9 s.