How to Build Scalable Operation Logs: AOP vs Binlog Implementation

Introduction

When users perform CRUD operations on important business data, we need to record their actions as operation logs for troubleshooting.

This article discusses common operation‑log types and compares two implementation schemes.

Common Operation Log Types

User login log

Important data query log (e.g., product search on e‑commerce platforms)

Important data change log (password, permission, data updates)

Data deletion log

…

Implementation Scheme Comparison

Traditional AOP‑based solution

Advantages: simple implementation idea.

Disadvantages: adds database load, strongly depends on front‑end parameters, hard to extend, no batch support, no multi‑table association.

Database Binlog‑based solution

Advantages: decouples old and new data, supports batch operations, easy multi‑table association, language‑agnostic.

Disadvantages: requires unified table design conventions.

Implementation Details

1. AOP + Annotation

Typical approach uses aspects and annotations to capture IP, module, user, scenario, source, etc. The following annotation example shows how new and old data are recorded.

@Valid
@NotNull(message = "New value cannot be null")
@UpdateNewDataOperationLog
private T newData;

@Valid
@NotNull(message = "Old value cannot be null")
@UpdateOldDataOperationLog
private T oldData;

Problems:

Old value must be supplied by the front‑end or an extra query, which may be stale.

Cannot handle batch List data.

No support for multi‑table operations.

When deleting, the previous value must be queried again.

@PostMapping("/delete")
@ApiOperation(value = "Delete user information", notes = "Delete user information")
@DeleteOperationLog(system = SystemNameNewEnum.SYS_JMS_LMDM,
    module = ModuleNameNewEnum.LMDM_AUTH,
    table = LogBaseTableNameEnum.TABLE_USER,
    methodName = "detail")

2. Binlog‑based solution

Architecture consists of three parts:

Business application generates a traceId for each operation, writes it to the business table, and sends a business message containing the operator information.

Log collector merges business logs with transformed binlog logs and provides a query API.

Log processor uses Canal to capture and parse binlog, pushes records to Kafka. The parsed record contains operation type, old and new values, e.g.:

{
  "data":[{"id":"122158992930664499","bill_type":"1","create_time":"2020-04-26 09:15:13","update_time":"2020-04-26 13:45:46","version":"2","trace_id":"exclude-f04ff706673d4e98a757396efb711173"}],
  "database":"yl_spmibill_8",
  "es":1587879945200,
  "id":17161259,
  "isDdl":false,
  "mysqlType":{"id":"bigint(20)","bill_type":"tinyint(2)","create_time":"timestamp","update_time":"timestamp","version":"int(11)","trace_id":"varchar(50)"},
  "old":[{"update_time":"2020-04-26 13:45:45","version":"1","trace_id":"exclude-36aef98585db4e7a98f9694c8ef28b8c"}],
  "pkNames":["id"],
  "sql":"",
  "sqlType":{"id":-5,"bill_type":-6,"create_time":93,"update_time":93,"version":4,"trace_id":12},
  "table":"xxx_transfer_bill_117",
  "ts":1587879945698,
  "type":"UPDATE"
}

After conversion, an operation log entry looks like:

{
  "id":"120716921250250776",
  "relevanceInfo":"XX0000097413282,",
  "remark":"Sign‑off financial outlet code changed from [] to [380000], ...",
  "traceId":"120716921250250775"
}

Database Table Design

All business tables must add a trace_id column. Log collector tables include table_config, table_field_config, and table_field_value. (SQL DDL omitted for brevity.)

Future Plans for Binlog Solution

Optimize business message sending by using aspects to reduce code intrusion.

Extend support for multi‑table association logging.

Conclusion

The article compares AOP and Binlog approaches for operation logging. AOP is simple and suitable for many small‑to‑medium projects, but it suffers from code intrusion and performance issues in high‑security or ERP systems. The Binlog method adds complexity but eliminates language dependence and intrusion, and it has been successfully deployed in production.

If you found this useful, please like and share.