Tagged articles
vectorization
48 articles · Page 1 of 1

TonyBaiOct 22, 2026 · Backend Development
This article explores Go's new experimental SIMD packages (simd/archsimd and simd), explaining SIMD hardware concepts, vector register operations, common SIMD patterns like broadcast and fused multiply-add, compiler intrinsic mapping, and demonstrates a dot product implementation achieving 9x speedup over scalar code.
AVX2FMAGo
0 likes · 31 min read

This article benchmarks seven common pandas tasks—arithmetic, conditional mapping, dictionary lookup, string cleaning, group aggregation, binning, and query/eval—showing how vectorized alternatives can speed up loops by up to 7,000× and offering practical guidance on when each technique delivers meaningful gains.
BenchmarkNumPyPandas
0 likes · 12 min read

This guide walks through adding embedding support to a Spring AI application by configuring Ollama, creating an EmbeddingService for vector generation and similarity calculations, exposing REST endpoints via EmbeddingController, and providing a simple HTML front‑end for interactive testing, with step‑by‑step instructions and code samples.
EmbeddingJavaOllama
0 likes · 19 min read

This article walks through why traditional keyword matching fails for RAG, explains the evolution from one‑hot encoding to Word2Vec and BERT, details sentence‑level embeddings and similarity metrics, compares leading Chinese and multilingual embedding models using the C‑MTEB benchmark, and provides practical LangChain code, deployment tips, and common pitfalls.
Chinese NLPEmbeddingLangChain
0 likes · 18 min read

This article explains why proper text chunking is critical for Retrieval‑Augmented Generation, illustrates common pitfalls with real‑world examples, compares four chunking strategies (fixed length, recursive, structure‑aware, and code‑aware), and provides practical guidelines for chunk size, overlap, metadata handling, and a production‑ready pipeline.
AI RetrievalLangChainRAG
0 likes · 21 min read

The article presents ten built‑in Pandas vectorized operations—such as np.select, assign, cut/qcut, melt/pivot_table, describe, query, transform, to_datetime, explode, and string accessor methods—showing concise one‑liners, their verbose equivalents, and the typical speed gains they deliver on large DataFrames.
Data ManipulationNumPyPandas
0 likes · 12 min read


ITPUBMar 27, 2026 · Databases
In 2026 the database ecosystem faces fierce competition between MySQL and PostgreSQL, while AI emerges as a new driver prompting open‑source projects like AliSQL to release DuckDB, vector engines and intelligent CLI, reshaping how relational databases serve both transactional and analytical workloads.
AIAliSQLDuckDB
0 likes · 15 min read

This article details Ant Group’s Flex vectorized engine built on Velox, covering the current state of vectorization, Flex’s architecture (Flink + Velox), core feature development, correctness guarantees, large‑scale deployment results, and future directions for full‑link vectorization and broader hardware support.
FlexFlinkVelox
0 likes · 18 min read

This guide explains how to run the Qwen3‑Embedding‑0.6B model on a CPU‑only server, configure key parameters, optionally use Intel Extension for PyTorch, and efficiently store the resulting vectors in Redis 8 with proper serialization and indexing.
CPUEmbeddingPython
0 likes · 8 min read

The article explains why deeply nested np.where calls hurt readability, speed, and maintainability, and demonstrates how using pandas assign() with boolean masks and loc can replace them, delivering 2‑4× faster execution on large DataFrames while keeping the logic clear and extensible.
NumPyPandasassign
0 likes · 14 min read

This article explores ClickHouse’s high‑performance OLAP design—including its MPP architecture, columnar storage, vectorized execution, pre‑sorting, sharding, replication, index strategies, and compute engine—showing how each innovation contributes to ultra‑fast, scalable data analysis in the big‑data era.
ClickHouseOLAPSharding
0 likes · 14 min read

This article explains the end‑to‑end implementation of Video RAG in OpenSearch LLM, covering offline parsing, key‑frame extraction, audio transcription, slice creation, multimodal vectorization, hybrid indexing, and online query processing while addressing challenges like recall performance and long‑video efficiency.
ASRKey Frame ExtractionLLM
0 likes · 10 min read

This article explains how to measure, analyze, and improve CPU usage in software by examining key metrics, choosing efficient algorithms and data structures, leveraging compiler options, exploiting cache behavior and SIMD vectorization, and applying practical case studies with Linux profiling tools.
CPUCacheGCC
0 likes · 31 min read

Didi replaced its legacy agent with Alibaba’s open‑source iLogTail, re‑architected it to use a shared thread‑pool and SIMD‑accelerated parsing, rewrote critical plugins in C++ and added robust Kafka retry logic, achieving over twice the throughput while cutting CPU usage by more than half and maintaining near‑zero latency at massive scale.
C++KafkaLog Collection
0 likes · 10 min read

The article introduces Blaze, Kuaishou's Rust‑powered native execution engine that vectorizes Spark SQL workloads, explains its architecture and operation, presents benchmark results showing up to 50% latency reduction, and details internal deployments, industry case studies, community collaborations, and the 2025 roadmap.
Native ExecutionRustSpark
0 likes · 12 min read

This article examines SIMD (Single Instruction Multiple Data) technology, its relevance to Go’s performance goals, the challenges of integrating SIMD into Go’s design, current standard‑library limitations, third‑party libraries, compiler support, and practical assembly examples, concluding with prospects for future Go SIMD adoption.
GoSIMDThird‑party libraries
0 likes · 15 min read

This article explains the challenges of big‑data processing in the cloud era, introduces Spark’s native‑language SQL engine rewrites, discusses vectorization and code generation techniques, describes cloud‑native storage‑compute separation with Remote Shuffle services such as Apache Celeborn, and presents the production benefits of Alibaba Cloud’s EMR Serverless Spark.
CodeGenEMR ServerlessRemote Shuffle
0 likes · 12 min read

This article introduces Flex, a Flink‑compatible stream‑batch vectorized engine built on Velox and Gluten, explains the SIMD‑based execution model, details native operator optimizations, fallback mechanisms, correctness and usability improvements, and presents performance results and future development plans.
FlinkSIMDVelox
0 likes · 17 min read

This article explains how Apache Doris achieves high‑concurrency OLAP and point‑query performance through MPP architecture, columnar storage, partition‑bucket pruning, various indexes, materialized views, vectorized execution, runtime filters, short‑circuit planning, and prepared‑statement caching.
DorisOLAPhigh concurrency
0 likes · 12 min read

This article introduces the AnalyticDB Spark architecture, explains the need for Spark vectorization, surveys industry vectorized solutions, details ADB Spark's own vectorized implementation with Gluten and Velox, and presents performance test results showing a 6.98‑fold speedup over open‑source Spark.
AnalyticDBGlutenSpark
0 likes · 9 min read

StarRocks combines vectorized execution, a new cost‑based optimizer, materialized views, a real‑time storage engine, pipeline execution, and distributed joins to deliver a unified, high‑performance OLAP solution that supports both traditional and lakehouse analytics while reducing operational complexity.
CBOOLAPStarRocks
0 likes · 14 min read

Blaze is Kuaishou's self‑developed native execution engine that leverages Rust, DataFusion, and SIMD vectorization to accelerate Spark workloads, offering a 30%+ compute boost, detailed architectural components, deep production‑grade optimizations, and a roadmap for broader adoption.
DataFusionNative ExecutionRust
0 likes · 13 min read

LibraDB, the column‑store replica of TDSQL MySQL, has evolved its execution engine from a simple scatter‑gather model to a vectorized SMP pipeline that integrates MPP parallelism, asynchronous I/O, SIMD‑accelerated aggregation and join operators, work‑stealing, and runtime filters, thereby fully exploiting CPU, memory, network and disk resources for both OLTP and analytical queries.
Execution EngineHash JoinMPP
0 likes · 22 min read

This article explains ClickHouse’s columnar, MPP‑based design, block compression, LSM pre‑sorting, sparse and skip‑list indexing, and vectorized execution, while also discussing its high‑frequency write challenges, concurrency limits, and production‑grade issues such as Zookeeper load and resource management.
ClickHouseColumnar DatabaseIndexing
0 likes · 11 min read

Meituan enhances Apache Spark by integrating the Gluten‑Velox vectorized execution engine, converting row‑wise operations to columnar SIMD processing, which yields over 40 % memory savings and up to 13 % faster runtimes across thousands of ETL jobs, while addressing stability, ORC support, shuffle redesign, and off‑heap memory optimization.
Apache SparkC++Gluten
0 likes · 30 min read


ITPUBApr 20, 2024 · Artificial Intelligence
This article uses a playful dialogue to demystify how large language models like GPT‑4 work, covering data collection, vectorization, the transformer’s attention mechanism, position encoding, training stages, multilingual translation, reasoning puzzles, and alignment, all illustrated through the tale of a curious learner named Wuming.
Attention MechanismTransformerartificial intelligence
0 likes · 50 min read

While both vectorization and hashing are essential for handling large datasets, this article examines whether vector embeddings can match the deterministic consistency of hash functions, comparing their collision handling, data structure design implications, and suitability for retrieval and machine‑learning tasks.
AIConsistencycollision handling
0 likes · 8 min read

This article presents a series of practical techniques—such as list comprehensions, pre‑computing lengths, using sets, skipping irrelevant iterations, inlining functions, generators, map, memoization, vectorization, and efficient string joining—that can accelerate Python for‑loops anywhere from 1.3‑fold up to 970‑fold, with concrete benchmark results and code examples.
Loop OptimizationPythonmemoization
0 likes · 15 min read

This article presents a comprehensive overview of precise deduplication in Meituan's Doris database, detailing the underlying bitmap data structures, aggregation bottlenecks, and a series of optimizations—including memory management, fast union, orthogonal encoding, and vectorized engine integration—that together achieve significant performance gains in high‑cardinality scenarios.
BitmapDorisOLAP
0 likes · 20 min read

The article explains how Huawei's BiSheng compiler enhances GaussDB performance through architecture‑level, module‑level, and function‑level optimizations such as inline expansion, instruction prefetch, auto‑vectorization, link‑time optimization, and feedback‑guided optimizations, and outlines future development plans.
BISHENGGaussDBInline
0 likes · 8 min read

This article presents Blaze, a Kuaishou‑built native execution middleware for SparkSQL that leverages Apache DataFusion to achieve vectorized operator execution, detailing its architecture, implementation, performance gains, current coverage, benchmark results, production rollout, and future development plans.
DataFusionNative ExecutionSpark
0 likes · 17 min read

This article explains how Pandas' vectorized string operations enable efficient, loop‑free processing of text data, covering basic methods like len() and lower(), advanced regex functions, and additional utilities such as split, replace, slice, and get_dummies, with code examples and usage details.
String Processingdata cleaningvectorization
0 likes · 21 min read

The article introduces the Gluten vectorized engine, explains why Spark’s CPU bottleneck motivates integrating native vectorized back‑ends via Substrait, details its architecture, component design, current performance gains of up to three‑fold, and outlines ongoing development and future work.
GlutenNative EngineSpark
0 likes · 18 min read

Xiaomi upgraded its Apache Doris from version 0.13 to the vectorized 1.1.2 release for its A/B experiment platform, conducting extensive single‑SQL and concurrent tests, identifying CPU, memory, and fragment timeout issues, and applying tuning such as memory decommit settings, string matching improvements, and patches to achieve up to 5× query speed gains and enhanced stability.
Apache DorisPerformance Tuningdatabase optimization
0 likes · 20 min read

This article explains the motivation, design, and implementation of vectorizing Apache Doris's storage layer using SIMD techniques, covering engine overview, vectorized programming concepts, storage architecture, index and predicate optimizations, delayed materialization, output improvements, and performance test results.
Apache DorisOLAPSIMD
0 likes · 13 min read

This article explains the definition of matrix‑to‑matrix derivatives, introduces the vectorization‑based differential approach using Kronecker products, presents key matrix‑vectorization properties, and walks through detailed examples illustrating how to compute such derivatives, highlighting their role and limitations in machine‑learning optimization.
Kronecker productderivativemachine learning
0 likes · 5 min read

The article presents a comprehensive technical overview of MercsDB—formerly HermesDB—including its background, storage and indexing designs, native and Presto computation engines, vectorization optimizations, benchmark results, real‑world applications, and future development plans.
MPPMercsDBbig data
0 likes · 20 min read

This article explains how CPU‑centric vectorization, especially SIMD, reduces instruction count and CPI, addresses the four major CPU bottlenecks, and how StarRocks systematically applies automatic and manual SIMD techniques, verification methods, and a suite of engineering optimizations to achieve multi‑fold query speedups.
CPU optimizationSIMDStarRocks
0 likes · 16 min read

An in‑depth visual tutorial explains NumPy’s core concepts—from one‑dimensional vectors to high‑dimensional tensors—covering array creation, indexing, arithmetic, broadcasting, sorting, and advanced functions like meshgrid and einsum, empowering developers to harness efficient multidimensional computations in Python.
NumPyPythonmultidimensional arrays
0 likes · 21 min read

This article explains how classic SQL execution engines use the volcano model and expression trees, discusses their performance drawbacks, introduces vectorized execution to reduce overhead, and describes Roaring Bitmap compression methods with container types for efficient storage and processing of integer sets.
Database EngineOperator TreeRoaring Bitmap
0 likes · 10 min read

This article demonstrates how to dramatically speed up the slow pandas apply function—by up to six hundred times—using Swifter for parallel execution, vectorized pandas/numpy operations, dtype downcasting, and direct .values array manipulation, with detailed timing comparisons.
Pandasapplydataframe
0 likes · 6 min read

This article explains how Apache Doris adopts CPU‑level vectorization and columnar storage to boost query performance, details the design and current status of its vectorized engine, and outlines future work such as JOIN acceleration, storage‑layer vectorization, import optimization, and extensive SQL function support.
Apache DorisSIMDSQL Engine
0 likes · 21 min read

This article explains how Apache Doris adopts CPU vectorization techniques—such as SIMD, columnar storage, and cache‑friendly designs—to boost query performance, detailing its current vectorized engine architecture, recent benchmarks, ongoing work on JOIN, storage, import, and future enhancements.
Apache DorisDatabase PerformanceSIMD
0 likes · 22 min read

The Fast OLAP Forum held on December 19 at DataFunCon gathers leading experts from Baidu, Tencent, JD, and FreeWheel to share cutting‑edge techniques in vectorized execution, cloud‑native ClickHouse, large‑scale OLAP architectures, and Presto optimizations, offering deep insights for practitioners dealing with massive real‑time data workloads.
Apache DorisClickHouseOLAP
0 likes · 7 min read

This article explains the various classifications of OLAP data warehouses—including MOLAP, ROLAP, HOLAP, and HTAP—based on data volume and modeling, reviews common open‑source ROLAP products, and details performance‑boosting techniques such as MPP architecture, cost‑based optimization, vectorized execution, and storage optimizations.
Cost-Based OptimizationMPPOLAP
0 likes · 27 min read

This article walks through several pandas performance‑boosting methods—from naive for‑loops and iterrows to apply, .isin, pd.cut, and NumPy digitize—showing timing results and demonstrating how vectorized operations can accelerate hourly tariff calculations by hundreds of times.
Pythondata-processingvectorization
0 likes · 10 min read

This article provides a comprehensive overview of OLAP systems, classifying them by data volume and modeling approach, comparing MOLAP, ROLAP, HOLAP and HTAP, reviewing popular open‑source products, and detailing architectural, query‑optimization, vectorization, storage and resource‑management techniques that affect analytical warehouse performance.
HTAPMOLAPOLAP
0 likes · 30 min read

The article explains how Spark 2.0’s second‑generation Tungsten engine replaces the traditional Volcano iterator model with whole‑stage code generation and vectorization, eliminating virtual calls, keeping temporary data in CPU registers, and using loop unrolling and SIMD to achieve order‑of‑magnitude performance gains on large‑scale data workloads.
Apache SparkTungstenWhole-stage code generation
0 likes · 12 min read
