Tagged articles

chunking strategy

4 articles · Page 1 of 1

Oct 15, 2026 · Artificial Intelligence

RAG Table Chunking: Why Rows Need Headers, Units, and Scoped Notes

The article explains why naive row-based table chunking for RAG loses critical context—headers, units, grouped rows, and scoped notes—and demonstrates a context-preserving approach that binds each row to its full explanatory metadata, enabling accurate retrieval without forcing models to guess meaning.

RAGchunking strategycontext preservation

0 likes · 18 min read

RAG Table Chunking: Why Rows Need Headers, Units, and Scoped Notes

Linyb Geek Road

Oct 5, 2026 · Databases

Beyond RAG: Vector Databases as General-Purpose Similarity Search Engines

This article explains why vector databases are general-purpose similarity search engines beyond RAG, covering embedding model selection (BGE), non-RAG use cases like recommendation and anomaly detection, core operations with ChromaDB and HNSW, selection criteria for ChromaDB, FAISS, Milvus, and Pinecone, and five practical engineering lessons on recall evaluation, API workarounds, distance vs similarity, chunking strategies, and metadata filtering.

ChromaDBEmbeddingHNSW

0 likes · 12 min read

Beyond RAG: Vector Databases as General-Purpose Similarity Search Engines

Fun with Large Models

Sep 23, 2026 · Artificial Intelligence

Building LLM Pre-training Data: Markdown Structured Chunking & Rule+LLM Quality Filtering

This guide details a complete pipeline for constructing LLM pre-training datasets: splitting Markdown documents into semantic chunks (1500 chars, no overlap) via heading-aware parsing, greedy packing, and oversized-unit handling, then filtering noise through rule-based checks (length, citations, symbols) and LLM-based quality judgment (completeness, relevance, cleanliness) to produce LLamaFactory-ready JSONL corpora.

LLM pre-trainingLLM-as-JudgeLlamaFactory

0 likes · 22 min read

Building LLM Pre-training Data: Markdown Structured Chunking & Rule+LLM Quality Filtering

Yunqi AI+

Mar 15, 2026 · Operations

How to Keep Your AI Knowledge Base Accurate and Up‑to‑Date

This guide explains the standardized workflow, documentation standards, and quality‑control mechanisms that business and product teams should follow to maintain a high‑quality AI knowledge base, ensuring agents always provide correct answers.

AI knowledge baseDocumentation StandardsFAQ creation

0 likes · 24 min read

How to Keep Your AI Knowledge Base Accurate and Up‑to‑Date