This article reviews the evolution of databases and large models, explains vector database fundamentals and RAG pipelines, and details Baidu's VectorDB architecture, performance advantages, and its role in AI‑enhanced database operations.
This article explores how large language models can transform data workflows by using embeddings for semantic search, building private domain knowledge bases, generating SQL code from natural language with visualized results, and enhancing exploratory data analysis, outlining practical steps and benefits for enterprises.
This guide explains how to build a Retrieval‑Augmented Generation (RAG) system using JD Cloud documentation as a knowledge base, storing document embeddings in ClickHouse, leveraging LangChain for vector retrieval, and exposing query and answer services via FastAPI and a Gradio UI.
This guide walks through creating a Retrieval‑Augmented Generation pipeline that harvests JD Cloud documentation, stores vector embeddings in ClickHouse, and serves queries via FastAPI, LangChain, a Qwen LLM, and a Gradio front‑end.
This article outlines ten critical components of a modern data lake reference architecture for AI/ML, detailing each function, the supporting vendor tools and open‑source libraries, and how they enable scalable storage, MLOps, distributed training, model hubs, vector search, and data visualization.
The article outlines LLM issues such as hallucination, outdated knowledge, and data privacy, then explains Retrieval‑Augmented Generation—detailing its data‑preparation and query‑time retrieval workflow, demonstrates a full LangChain implementation, and contrasts RAG with fine‑tuning as complementary strategies for up‑to‑date, grounded responses.
This article walks through the practical use of JD’s self‑developed Vearch vector database—covering cluster creation, space setup, data insertion, and both text and vector search—illustrating how it integrates with LangChain and OpenAI embeddings to enable retrieval‑augmented generation for large language models.
This article explains the fundamentals and engineering practices of large language models (LLM), retrieval‑augmented generation (RAG) and AI agents, compares small and large embedding models, provides Python code for vector‑database RAG with Chroma, and discusses integration, use cases, and future challenges in AI development.
This article describes how large language models and the LangChain framework can be combined with PDF parsing, text chunking, memory management, and a vector database to automatically generate software test cases, achieving significant efficiency gains while outlining implementation details, results, and future challenges.
Spring AI brings artificial‑intelligence capabilities to the Spring Boot ecosystem, offering model support, vector‑database integration, SQL‑like filtering, and easy Maven configuration, enabling Java developers to add generative AI, semantic search, and AI‑driven image generation to their backend applications.
This guide explains how to implement a Retrieval‑Augmented Generation (RAG) workflow by loading a beer‑info JSON dataset into a Redis vector store, wiring it with Spring Cloud Alibaba AI Starter, and exposing a web API that returns LLM‑generated answers.
This guide explains ByteHouse’s high‑performance vector search capabilities, covering the background of vector retrieval for LLMs, the limitations of its existing skip‑index architecture, the new vector‑index design with HNSW and IVF, query‑time optimizations, performance benchmarks against Milvus, and future development plans.
The talk explores how the rapid growth of multimodal data and large language models is reshaping data processing, highlighting three key trends—online‑offline integration, vector‑relational database convergence, and the fusion of data processing with AI computation—while presenting practical solutions and future visions for unified data‑AI ecosystems.
With the rise of LLMs, vector search and vector databases have become essential for extending model memory, and this article explains the principles, algorithms, design choices, implementation details, and performance results of ByteHouse’s cloud‑native vector retrieval engine.
VikingDB is a cloud‑native vector database that originated at ByteDance, offering high‑performance ANN search, hybrid dense‑sparse retrieval for Retrieval‑Augmented Generation, extensive scaling and filtering capabilities, and ready‑to‑use SDKs for real‑world AI applications.
This tutorial walks through building a Retrieval‑Augmented Generation application by setting up Baidu’s Vector Database and Qianfan embedding service, configuring credentials, creating a document database and vector table, loading and chunking PDFs, generating embeddings, storing them, and performing scalar, vector and hybrid similarity searches, ready for integration with Wenxin LLM for answer generation.
This guide compares Ollama and Hugging Face for running large language models locally, detailing API and local execution methods, installation steps, model selection, resource requirements, integration with AnythingLLM, container deployment, embedding and vector store setup, and practical observations on performance and limitations.
Dolphin VectorDB, created by Alibaba’s Alimama team, is a high‑performance, scalable vector database that delivers fast, cost‑effective AI‑driven vector storage and real‑time updates, supporting multiple query modes and powering applications such as content risk control, marketing Q&A, and audience selection, with ongoing enhancements for multimodal computing.
ChromaDB is an open‑source, AI‑native vector database that efficiently stores, indexes, and retrieves high‑dimensional embeddings, offering fast similarity search, easy integration via flexible APIs, strong scalability, and active community support, making it suitable for recommendation systems, NLP, and image‑recognition workloads.
The article explains Retrieval-Augmented Generation (RAG), its workflow, advantages, comparison with fine-tuning, and provides a step-by-step implementation using Baidu's ERNIE SDK, LangChain, and ChromaDB to build a personal knowledge base that answers queries with retrieved context.
This article reviews the author’s hands‑on experience with Pinecone’s serverless vector database, various embedding and generation models such as all‑MiniLM‑L6‑v2, text‑embedding‑ada‑002, clip‑ViT‑B‑32, and GPT‑3.5‑turbo‑instruct, and demonstrates how they are applied to semantic search, RAG, recommendation, hybrid, and facial similarity tasks using Python code examples.
The paper details Baidu Intelligent Cloud’s design and deployment of a domain‑specific knowledge‑base Q&A system for database operations, combining prompt‑engineered LLMs with hybrid vector‑search using LangChain, BES vector store, and custom ingestion, addressing recall, token limits, and hallucination challenges across dashboard and IM bot interfaces.
This article details Baidu Intelligent Cloud's database operations team’s end‑to‑end design of an AI‑driven knowledge‑base Q&A system, covering background, architecture, technical choices, module implementation, key challenges such as vector‑search recall and token limits, and real‑world deployment scenarios.
This article explores the emerging enterprise knowledge assistant paradigm in the era of large models, detailing traditional knowledge management challenges, solution architecture using vector databases and LLMs, core technologies such as ETL pipelines, reranking, secure fine‑tuning, and future prospects for intelligent enterprise applications.
This article traces the 70‑year evolution of databases, examines how the rise of AIGC, cloud computing and AI native architectures are reshaping the industry, and details Baidu Smart Cloud's historical milestones, flagship products such as GaiaDB and PegaDB, and the emerging trends that will drive the next generation of database solutions.
The article examines AutoGPT's shift away from vector databases, detailing the original vision of using embeddings for long‑term memory, the performance calculations that exposed unnecessary complexity, the adoption of JSON‑based storage, and the emerging trend of specialized multi‑agent architectures.
Vector databases gain traction because they dramatically reduce storage, learning, scaling, and large‑model limitations costs by enabling semantic similarity search, RAG‑based prompt optimization, efficient high‑dimensional indexing, and cloud‑native architectures, making them essential for modern AI applications despite the promotional context.
This article outlines Douyin’s step‑by‑step engineering evolution of its cloud‑native vector database, covering the background of vector search, core concepts, algorithmic optimizations, storage‑compute separation, streaming updates, multi‑tenant orchestration, and future applications such as large language model integration.
This article explores how vector databases serve as the memory layer for large AI models, detailing their distributed, compute‑separated architecture, performance optimizations, hybrid vector‑scalar retrieval, and practical deployments across TikTok’s ecosystem such as image search, intelligent Q&A, and multimodal AI services.
This article explains the design and implementation of the Codebase AI Assistant in the Chocolate Factory framework, covering its problem‑solving DSL, retrieval‑augmented generation pipeline, indexing and querying stages, prompt strategies, and code‑splitting rules that together enable efficient semantic code search.
This tutorial explains how to leverage generative AI tools—including LLMs, embedding models, vector databases, and the LangChain framework—to create a custom chatbot that answers user queries using a knowledge base, with step‑by‑step code examples for Google Colab.
This article introduces Milvus, an open‑source, cloud‑native vector database designed for AI workloads, explains how it helps mitigate large‑model hallucinations, outlines its CVP architecture, showcases performance benchmarks, and explores diverse application scenarios and future directions for LLM‑vector database integration.
This article demystifies vector databases by first explaining the concept with a five‑year‑old analogy, then expanding to technical details for developers, covering how embeddings work, the differences from relational databases, ANN search, indexing, similarity metrics, and why vector stores outperform raw NumPy arrays for large‑scale similarity retrieval.
The article analyzes how enterprise data systems must be re‑engineered for large‑model applications, outlines the three‑stage data pipeline (ingestion, orchestration, interaction), introduces data‑virtualization techniques with virtual tables and intelligent materialization, and validates the approach with two banking case studies.
The guide explains GPT’s core capabilities—natural language mastery, domain reasoning, and code generation—while detailing its limits such as prompt sensitivity, token caps, and lack of memory, then offers engineering workarounds like systematic prompting, chain‑of‑thought, external memory, tool integration, safety checks, and a six‑layer architecture for building robust commercial AI applications.
The 14th China Database Technology Conference (DTCC2023) showcased cutting‑edge advances in vector databases, data privacy, MySQL security, and AI‑driven intelligent operations, featuring insights from industry leaders at Huawei, Tencent, eBay, Bilibili and more.
The article explains how combining vector databases with large language models can help governments and enterprises leverage massive private data for AI, covering semantic search, approximate nearest neighbor techniques, alignment challenges across modalities, and future directions for fine‑grained data integration.
This article explains how to build a text‑to‑image search system by first extracting text with OCR, then storing image paths and textual embeddings in a SQLite or Milvus vector database, and finally improving retrieval with Transformer‑based sentence embeddings and image‑captioning models.
The author builds an internal code knowledge base for the TDesign Vue‑Next library by scraping documentation, chunking and embedding texts with OpenAI’s ada model into a vector store, then retrieving relevant chunks for LLM answers, and enhances context continuity using a JavaScript AST interpreter, achieving up to 90 % query accuracy and a 20 % productivity boost.
This article explains the fundamentals of vector retrieval, its role in enhancing large language models through embedding and prompt engineering, and details the algorithms, system architecture, and Baidu's engineering practices for building high‑performance vector databases.
This article explains what vectors and vector databases are, how they differ from traditional databases, their core technologies, their relationship with large language models, market trends, and details of Tencent Cloud VectorDB’s capabilities, architecture, real‑world applications, and future competitive challenges.
Vector databases store and search large-scale vector data, and in 2023 the five leading solutions—Chroma, Pinecone, Weaviate, Milvus, and Faiss—offer scalable, high-performance options for applications such as LLM-driven services, audio search, recommendation systems, image/video analysis, and semantic retrieval across various industries.
The article explains what vector databases are, how vectors and embeddings work, the main embedding methods such as matrix factorization, NLP and graph techniques, the characteristics and high‑availability requirements of vector databases, and common AI‑driven application scenarios like semantic search, recommendation and anomaly detection.
This article outlines a reference stack for LLM applications, detailing the context‑learning design pattern, key components such as vector databases and embedding models, orchestration frameworks like LangChain, and discusses trade‑offs between proprietary and open‑source models, scaling challenges, and the future role of AI agents.
This article explains what embeddings and vector databases are, why they matter for large language models, and how they can be used to overcome token limits by storing and retrieving relevant text chunks for accurate, context‑aware responses.
This article explains why combining large language models with vector databases enables enterprises to create specialized, up‑to‑date chatbots, outlines the underlying principles, describes the ADB‑PG vector‑search capabilities, and provides step‑by‑step implementation details including data processing, indexing, and query examples.
This article examines Auto‑GPT’s hype versus reality, detailing its architecture, cost per task, looping pitfalls, limited memory management, and why its current capabilities fall short of practical production use.
Prompt‑Ops frameworks like LangChain let developers turn pre‑trained LLMs into versatile applications by abstracting model calls, chaining prompts, integrating tools, managing memory, and handling private data, while addressing challenges such as nondeterminism, version control, and prompt injection in production environments.
Milvus 2.1, the newly released open‑source vector database, brings trillion‑byte storage, sub‑5 ms search latency, and seamless scalability for both structured and unstructured data, positioning it as a game‑changing infrastructure for similarity search across industries.
This article presents a complete image‑search solution that extracts visual features with the VGG16 model, stores them in the Milvus vector database, and provides a set of web APIs for training, querying, counting, searching, and deleting image vectors, all deployed via Docker containers.
This article explains how the Milvus vector database and the Towhee embedding framework together enable large‑scale, high‑throughput semantic analysis and retrieval of unstructured data such as images, video, and audio by leveraging AI‑powered vectorization and search pipelines.
This article walks through building a large‑scale semantic search system on the English Wikipedia using the Weaviate vector database, GraphQL queries, and pre‑trained Sentence‑BERT and BERT Q&A models, covering dataset preparation, schema design, import pipelines, query examples, and production deployment strategies.
The article explains why traditional text‑based search engines like ElasticSearch struggle with modern multimodal data, introduces vector databases that store implicit semantic embeddings, and proposes a generalized search architecture that decouples data‑to‑vector mapping from the engine while leveraging clustering or graph indexes for similarity search.
This article walks through the complete engineering workflow for building an image‑search system, covering CNN‑based feature extraction with VGG16, vector normalization, image preprocessing, black‑edge removal, and practical deployment of the Milvus vector database including hardware requirements, capacity planning, collection/partition design, and search result handling.
