Big Data Technology & Architecture

May 28, 2020 · Big Data

From SQL to RDD: Understanding Spark's Internal Architecture

This article explains how Spark converts SQL queries into RDD operations by creating a SparkSession, registering temporary views, executing SQL, and then detailing the underlying InternalRow, TreeNode, and Expression structures that power the Catalyst optimizer.