Using MySQL 8.0 JSON Error Log Output and Filtering
MySQL 8.0 adds JSON‑formatted error logs and flexible filtering components, allowing administrators to install JSON sink and filter modules, extract specific fields with tools like jq, and apply rules that drop or throttle messages to improve log readability and automation.
MySQL 8.0 redefines error‑log output by adding optional JSON formatting and a new filtering mechanism, which makes the previously bulky and hard‑to‑read logs much more structured and machine‑friendly.
An example JSON log file ( mysqld.log.00.json ) shows fields such as log_type , prio , err_code , subsystem , msg , time , and others for each error record.
After installing the JSON sink component with INSTALL COMPONENT 'file://component_log_sink_json'; and enabling it via SET GLOBAL log_error_services = 'log_filter_internal; log_sink_json'; , the error log is written both to the traditional text file and to a JSON file, which can be queried with jq . mysqld.log.00.json or by extracting specific fields like jq '.msg' mysqld.log.00.json .
To filter logs further, the component_log_filter_dragnet component is installed ( INSTALL COMPONENT 'file://component_log_filter_dragnet'; ) and the global filter rules are set, for example SET GLOBAL dragnet.log_error_filter_rules = 'IF prio>=WARNING THEN drop.'; to keep only error messages, or SET GLOBAL dragnet.log_error_filter_rules = 'IF prio==WARNING THEN throttle 1/60.'; to allow at most one warning per 60 seconds.
Test queries that reference missing tables generate errors (e.g., ERROR 1812 (HY000): Tablespace is missing for table `ytt2`.`a`. ), which appear as five records in both the plain‑text log and the JSON log. After applying the filter rules, only the desired error entries remain, while warnings are either dropped or throttled according to the configured policy.
In summary, MySQL 8.0’s component_log_sink_json and component_log_filter_dragnet provide a powerful and flexible way to output, parse, and filter error logs in JSON, enabling easier automation and analysis of database issues.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.