Tagged articles
DataFrames
18 articles · Page 1 of 1
The article shows that Polars, a query‑compiling DataFrame library, can accelerate ten‑million‑row GroupBy workloads by 6‑10× compared with Pandas, explains the underlying optimizer, Arrow columnar engine and Rust parallelism, provides a 20‑item syntax map, three real migration scenarios, streaming for out‑of‑memory data, and AI‑pipeline use cases, and offers a step‑by‑step migration guide.
Apache ArrowDataFramesPandas
0 likes · 21 min read

Polars is a high‑performance DataFrame library written in Rust with a Python API, offering lightning‑fast operations, zero‑copy I/O, parallel execution, lazy evaluation, and seamless integration with Arrow, making it a compelling alternative to pandas for large‑scale data processing.
DataFramesPolarsPython
0 likes · 22 min read

This article compares Polars with Pandas, Tidyverse, and Base R, showing how Polars’ Rust implementation, Arrow integration, lazy execution and expressive API deliver 5‑10× speed gains and lower memory usage, and explains why beginners may find Polars easier to learn.
ArrowDataFramesLazy Execution
0 likes · 17 min read

This article introduces PySpark as the Python API for Apache Spark, explains Spark's core concepts and advantages, details PySpark's main components and a simple code example, compares it with Pandas, and outlines typical big‑data scenarios and further learning directions.
Apache SparkDataFramesPySpark
0 likes · 5 min read

This tutorial walks through two practical ways to sum corresponding columns from two pandas DataFrames in Python, handling mismatched column names and showing both a rename‑and‑add approach and a NumPy‑based solution with clear code examples.
DataFramesPythoncolumn addition
0 likes · 3 min read

This article walks through a Python community question by presenting two clear approaches—renaming columns to match and using NumPy addition—to sum corresponding columns from two tables, complete with visual code examples and a brief discussion on handling mismatched headers.
DataFramesNumPyPandas
0 likes · 3 min read

This article teaches how to use Pandas for data reshaping, covering the melt method for converting wide-format data to long-format and the pivot method for the reverse transformation, with practical code examples.
DataFramesPivotPython
0 likes · 7 min read

This article explains the differences between pandas loc/iloc and at/iat, demonstrates why using loc/iloc inside loops can be extremely slow, and shows how replacing them with at/iat can speed up DataFrame updates by up to 60 times.
A*DataFramesPandas
0 likes · 6 min read

This article introduces cuDF, a Python GPU DataFrame library with a pandas‑like API, compares it to pandas, explains when to use cuDF versus Dask‑cuDF for single‑GPU or multi‑GPU workloads, and provides practical code examples for common data operations.
DataFramesGPUPython
0 likes · 7 min read

This article provides a comprehensive overview of NumPy and pandas, covering ndarray basics, multi‑dimensional array creation, core array attributes, broadcasting, random number generation, reshaping, as well as pandas Series and DataFrame structures, data import/export, grouping, merging, and advanced data manipulation techniques for scientific and data‑analysis tasks.
Array OperationsDataFramesNumPy
0 likes · 17 min read

This article provides a detailed exploration of SparkSQL, covering its evolution from Shark, core components, execution workflow, Catalyst optimizer, various optimization strategies, and practical configuration tips for achieving high performance in big‑data processing.
Catalyst OptimizerDataFramesPerformance Tuning
0 likes · 19 min read

This article explains how Spark SQL processes joins—including inner, outer, semi, and anti joins—by describing the overall query planning flow, the three physical join strategies (sort‑merge, broadcast, and hash), and the specific implementation details for each join type.
DataFramesSQL OptimizationSpark SQL
0 likes · 10 min read

This tutorial walks you through using Python and Pandas to import Excel files, explore DataFrames, apply filters, perform statistical calculations, create pivot tables, and emulate Excel functions like VLOOKUP, providing code examples and visual guides for each step.
DataFramesExcelPandas
0 likes · 9 min read

This article introduces Spark SQL fundamentals, including its architecture, DataFrame and Dataset abstractions, query methods, interoperability with RDD, user-defined functions, integration with Hive, data source handling, and provides step‑by‑step Scala code examples for loading data, performing aggregations, and solving common analytical tasks.
DataFramesHiveSQL
0 likes · 15 min read

This article details how Suning built a Hadoop‑based big data platform and leveraged Apache Spark to process terabytes of product price and inventory data, describing the system architecture, four key technical practices, performance results, and future data‑lake directions.
Apache SparkDataFramesETL
0 likes · 12 min read

This article provides an in‑depth technical overview of Apache Spark, covering its core concepts such as RDDs, transformation and action operations, execution models, Spark 2.0 enhancements like unified DataFrames/Datasets, whole‑stage code generation, Structured Streaming, and practical performance‑tuning guidance.
DataFramesRDDSpark
0 likes · 20 min read

This article reviews Spark's evolution from version 1.2 to 1.6, explains the DataFrame and Tungsten projects, shares Hulu’s real‑world Spark deployments, and discusses performance‑related challenges such as stack overflow, streaming receiver latency, and class‑loader deadlocks.
DataFramesDataset APIHulu
0 likes · 17 min read

This article provides a comprehensive overview of Apache Spark, covering its origins, core concepts such as RDDs, transformations, actions, dependencies, execution modes, and key components like Spark SQL, Streaming, MLlib, and GraphX, while also offering practical code examples and visual illustrations.
DataFramesGraphXMLlib
0 likes · 18 min read
