Understanding MySQL SQL Parsing: Yacc Grammar, Parse Trees, and Call Stacks

The article introduces the process of parsing an SQL statement in MySQL, emphasizing how mastering SQL syntax analysis can enable tasks such as SQL auditing and routine checks.

It explains that syntax analysis generates an abstract syntax tree (AST) and that MySQL uses Yacc (Yet Another Compiler Compiler) to convert tokenized SQL into an AST. Yacc consists of a definition section, a rule section, and user subprograms.

Yacc Grammar Structure

Symbols (tokens) are strings of letters, digits, periods, or underscores that do not start with a digit. Tokens produced by the lexical analyzer are called terminal symbols or tokens. Non‑terminal symbols appear on the left side of rules.

Definition Section

The definition section contains literal text copied to the generated C file, including token definitions such as %union, %start, %token, %type, %left, %right, %nonassoc, and optional C code enclosed in %{ … %}.

Rule Section

The rule section defines grammar rules and associated C actions. Each rule places the non‑terminal on the left, followed by a colon, the production (body), and an action block enclosed in braces.

Example

MySQL's Yacc description file is located at sql/sql_yacc.yy .

Parse‑Tree Related Declaration and Definition Files

parse_tree_column_attrs.h  parse_tree_hints.cc  parse_tree_node_base.h
parse_tree_handler.cc     parse_tree_hints.h   parse_tree_nodes.cc
parse_tree_handler.h      parse_tree_items.cc  parse_tree_nodes.h
parse_tree_helpers.cc     parse_tree_items.h   parse_tree_partitions.cc
parse_tree_helpers.h      parse_tree_node_base.cc  parse_tree_partitions.h

Parse‑tree node base class: Parse_tree_root .

Other Parse‑Tree Declarations

item.cc               item_geofunc_internal.h   item_row.cc
item.h                item_geofunc_relchecks.cc item_row.h
item_buff.cc          item_geofunc_relchecks_bgwrap.cc item_strfunc.cc
item_cmpfunc.cc       item_geofunc_relchecks_bgwrap.h  item_strfunc.h
item_cmpfunc.h        item_geofunc_setops.cc   item_subselect.cc
item_create.cc       item_inetfunc.cc         item_sum.cc
item_create.h        item_inetfunc.h          item_sum.h
item_func.cc         item_json_func.cc        item_timefunc.cc
item_func.h          item_json_func.h         item_timefunc.h
item_geofunc.cc      item_pfs_func.cc        item_xmlfunc.cc
item_geofunc.h       item_pfs_func.h         item_regexp_func.h
item_geofunc_buffer.cc  item_regexp_func.cc   item_xmlfunc.h   item_geofunc_internal.cc

Common SQL Error Messages

Parse‑stage errors (e.g., syntax errors):

ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '123' at line 1

Prepare‑stage errors (e.g., missing table):

ERROR 1146 (42S02): Table 'ccfdb.x' doesn't exist

When encountering errors, the article suggests checking share/messages_to_clients.txt for further details.

SQL Parsing Call Stack

#1    ./sql/conn_handler/connection_handler_per_thread.cc::handle_connection
#2.1  ./sql/sql_parse.cc::bool do_command(THD *thd)
#2.2  ./sql/sql_parse.cc::dispatch_command
#3    ./sql/sql_parse.cc::mysql_parse
#4.1  ./sql/sql_lex.cc:bool lex_start(THD *thd)
#4.2  ./sql/sql_parse.cc::parse_sql
#5    ./sql_class.cc:bool THD::sql_parser()
#6.1  ./sql_yacc.cc:int MYSQLparse
#6.2  ./sql_lex.cc:bool LEX::make_sql_cmd(Parse_tree_root *parse_tree)

Example of inspecting table and database names is shown with a screenshot (omitted here).

SQL Pre‑Parsing Call Stack

#1 sql/sql_parse.cc:int mysql_execute_command(THD *thd, bool first_level)
#2 sql/sql_select.cc:bool Sql_cmd_dml::execute(THD *thd)
#3 sql/sql_select.cc:bool Sql_cmd_dml::prepare(THD *thd)
#4 sql/sql_select.cc:bool Sql_cmd_select::precheck(THD *thd)
#5 auth/sql_authorization.cc:bool check_table_access
#6 auth/sql_authorization.cc:bool check_grant

Finally, the article displays the parse tree of a sample SELECT statement.