Tagged articles
Arrow
12 articles · Page 1 of 1
The article explains how lance‑spark enables Lance's Blob V2 storage by marking a column with blob encoding and setting file_format_version ≥ 2.2, describes the metadata‑driven descriptor schema, the write path that still accepts Spark BINARY, and the size‑based placement of binary data into inline, packed, dedicated or external blob files.
ArrowBlob V2Spark
0 likes · 12 min read

Apache Spark 4.2 introduces a lightweight Spark Connect architecture, native AI integration, enhanced Metrics View for unified semantics, Arrow‑first performance gains, advanced SQL extensions like vector search and QUALIFY, robust geospatial support, and a revamped streaming engine with auto CDC and sub‑millisecond state cleanup.
Apache SparkArrowCDC
0 likes · 13 min read

This article compares Polars with Pandas, Tidyverse, and Base R, showing how Polars’ Rust implementation, Arrow integration, lazy execution and expressive API deliver 5‑10× speed gains and lower memory usage, and explains why beginners may find Polars easier to learn.
ArrowDataFramesLazy Execution
0 likes · 17 min read

This article shares detailed practical experiences from DingXiangYuan's big‑data platform on using Apache Kyuubi and Apache Celeborn, covering architecture, flexible configuration, AuthZ fine‑grained permissions, small‑file and Z‑Order optimizations, Arrow‑based large result transmission, and operational tips such as connection‑level issues and Netty cache handling.
Apache CelebornApache KyuubiArrow
0 likes · 17 min read

This article provides a thorough overview of Python's time‑related libraries—including the built‑in time and datetime modules, third‑party pytz, dateutil, and Arrow—explaining core concepts such as timestamps, time zones, daylight‑saving time, and ISO 8601 parsing with clear code examples and usage patterns.
ArrowDATETIMEDateUtil
0 likes · 27 min read

This article presents a comprehensive technical overview of how DXY's big data platform leverages Apache Kyuubi and Celeborn to unify Spark entry points, configure flexible task isolation, implement fine‑grained AuthZ, optimize small files and Z‑Order sorting, and accelerate large result set transmission with Arrow, while also discussing operational challenges and upcoming features.
Apache KyuubiArrowConfiguration
0 likes · 17 min read

The article details China Agricultural Bank’s ARROW platform achieving third‑level DevOps continuous delivery certification, outlining its end‑to‑end pipeline, quality gates, metric‑driven improvements, and how these practices boost code quality, delivery speed, and support the bank’s digital transformation.
ArrowContinuous DeliveryDevOps
0 likes · 8 min read

This comprehensive guide walks you through Python’s time‑handling ecosystem, covering the low‑level time module, the richer datetime module, timezone utilities like pytz, advanced parsing with dateutil, and the user‑friendly Arrow library, complete with explanations, code snippets, and practical examples for formatting, conversion, and manipulation.
ArrowDATETIMEDateUtil
0 likes · 39 min read

This article provides a detailed overview of Python's time‑handling ecosystem, covering fundamental concepts of seconds, GMT, UTC and leap seconds, the low‑level time module, high‑level datetime classes, timezone manipulation with pytz, flexible parsing with dateutil, and the user‑friendly Arrow library, complete with code examples and usage notes.
ArrowDATETIMEDateUtil
0 likes · 24 min read

This article introduces Python's built‑in time modules, explains why third‑party libraries like DateParser, Dateutil, and Arrow are needed for unconventional date formats, and demonstrates how to use them—including code examples—to split time ranges by month or week for robust data export tasks.
ArrowDate ParsingDateUtil
0 likes · 6 min read

This article provides a step‑by‑step guide on using the Python Arrow library to obtain UTC and local times, format dates, shift time values, parse arbitrary timestamps, and convert Arrow objects to native datetime, date, and time types, complete with code examples and result screenshots.
ArrowDATETIMEPython
0 likes · 5 min read

This article explains PySpark's multi‑process architecture, how the Python driver uses Py4J to call Java/Scala APIs, the implementation of RDD and DataFrame interfaces, executor‑side process communication and serialization with Arrow, and the design of Pandas UDFs, while also discussing current limitations and future directions.
ArrowPySparkPython
0 likes · 13 min read
