This article explains Lance's Stable Row ID feature, which assigns persistent logical identifiers to rows so they survive compaction and updates, reducing index maintenance costs at the expense of extra mapping storage and lookup overhead.
At AICon 2026, Ma Jin explained how Lance’s layered lakehouse format—spanning file, table, index and catalog layers—addresses AI workloads by supporting wide columns, cheap schema evolution, mixed storage and low‑latency random reads, enabling unified handling of training data, RAG retrieval and Agent memory.
This article analytically compares three open table‑format projects—Paimon, Hudi, and Iceberg—examining how each addresses multimodal data lake challenges such as massive volume, sparse access patterns, and combined scalar‑vector retrieval, and provides concrete feature breakdowns and selection guidance.
The article walks through Lance’s describe_indices() workflow—from reading the manifest and caching index metadata, through optional filtering and grouping by logical index name, to building human‑readable index descriptions and highlighting differences from load_indices and index_statistics, while noting edge cases and limitations.
The article explains the purpose and inner workings of Lance's optimize_indices function, detailing how it incorporates newly appended data into existing indexes, merges delta indexes, and manages partition adjustments to maintain fast vector and scalar query performance without full re‑training.
AliSQL 8.0 introduces a shared Nodes Cache and per‑transaction cache to speed up vector queries, implements RC‑level transaction isolation for read‑only and read‑write operations, and leverages SIMD‑based pre‑computation to dramatically improve high‑dimensional vector distance calculations and concurrency performance.
This article explains the core concepts of vector database indexing, details the composition of Milvus indexes—including data structures, quantization methods, and specific algorithms like IVF, HNSW, DISKANN, PQ, RABITQ, PRQ, SCANN, AISAQ, and MINHASH_LSH—and offers speculation on future developments.
This article examines the high cost and low throughput of embedding vectors in log‑processing scenarios, analyzes the performance bottlenecks of inference frameworks, and details a series of cloud‑native optimizations—including switching to vLLM, deploying multiple model replicas with Triton, decoupling tokenization, and priority queuing—that together raise throughput by 16× and reduce per‑token pricing by two orders of magnitude.
This article details the cost and speed challenges of embedding vectors in large‑scale log scenarios, analyzes inference framework choices, describes GPU utilization, priority queuing, and pipeline redesigns, and reports a 16‑fold throughput increase and dramatically lower per‑request costs.
