This article compares Polars with Pandas, Tidyverse, and Base R, showing how Polars’ Rust implementation, Arrow integration, lazy execution and expressive API deliver 5‑10× speed gains and lower memory usage, and explains why beginners may find Polars easier to learn.
This article shares detailed practical experiences from DingXiangYuan's big‑data platform on using Apache Kyuubi and Apache Celeborn, covering architecture, flexible configuration, AuthZ fine‑grained permissions, small‑file and Z‑Order optimizations, Arrow‑based large result transmission, and operational tips such as connection‑level issues and Netty cache handling.
This article provides a thorough overview of Python's time‑related libraries—including the built‑in time and datetime modules, third‑party pytz, dateutil, and Arrow—explaining core concepts such as timestamps, time zones, daylight‑saving time, and ISO 8601 parsing with clear code examples and usage patterns.
This article presents a comprehensive technical overview of how DXY's big data platform leverages Apache Kyuubi and Celeborn to unify Spark entry points, configure flexible task isolation, implement fine‑grained AuthZ, optimize small files and Z‑Order sorting, and accelerate large result set transmission with Arrow, while also discussing operational challenges and upcoming features.
The article details China Agricultural Bank’s ARROW platform achieving third‑level DevOps continuous delivery certification, outlining its end‑to‑end pipeline, quality gates, metric‑driven improvements, and how these practices boost code quality, delivery speed, and support the bank’s digital transformation.
This comprehensive guide walks you through Python’s time‑handling ecosystem, covering the low‑level time module, the richer datetime module, timezone utilities like pytz, advanced parsing with dateutil, and the user‑friendly Arrow library, complete with explanations, code snippets, and practical examples for formatting, conversion, and manipulation.
This article provides a detailed overview of Python's time‑handling ecosystem, covering fundamental concepts of seconds, GMT, UTC and leap seconds, the low‑level time module, high‑level datetime classes, timezone manipulation with pytz, flexible parsing with dateutil, and the user‑friendly Arrow library, complete with code examples and usage notes.
This article introduces Python's built‑in time modules, explains why third‑party libraries like DateParser, Dateutil, and Arrow are needed for unconventional date formats, and demonstrates how to use them—including code examples—to split time ranges by month or week for robust data export tasks.
This article provides a step‑by‑step guide on using the Python Arrow library to obtain UTC and local times, format dates, shift time values, parse arbitrary timestamps, and convert Arrow objects to native datetime, date, and time types, complete with code examples and result screenshots.
This article explains PySpark's multi‑process architecture, how the Python driver uses Py4J to call Java/Scala APIs, the implementation of RDD and DataFrame interfaces, executor‑side process communication and serialization with Arrow, and the design of Pandas UDFs, while also discussing current limitations and future directions.
