vivo Internet Technology
Jan 31, 2018 · Big Data
Predicate Pushdown Rules in SparkSql Inner Join Queries
SparkSql optimizes inner‑join queries by pushing predicates to the scan phase, allowing filters connected with AND to be applied before the join without changing results, while OR‑connected filters can be unsafe except when they involve the join key or partitioned tables which use partition pruning.
Join OptimizationPartition PruningPredicate Pushdown
0 likes · 10 min read