Apache Fluss Virtual Tables: $changelog vs $binlog for Queryable Change Streams

What Are Virtual Tables?

Virtual tables in Fluss are system-generated, read-only tables that do not store additional data. Instead, they expose the underlying table's change log (Change Log) in real time. Users access them by appending a specific suffix to the base table name: base_table_name + $suffix → virtual table.

Applicability quick reference: Log Tables support only $changelog; PrimaryKey Tables support both $changelog and $binlog. Because Log Tables are append-only and never modify data, exposing insert events alone fully represents their change history.

Changelog Virtual Table ( $changelog )

2.1 Use Cases

Real-time auditing – track every change history of each record.

ETL incremental sync – capture changes and synchronize to downstream systems.

Event sourcing – rebuild state from an event stream.

2.2 Schema Structure

$changelog

prepends three metadata columns before all original columns of the base table (the base table schema remains identical). The three metadata columns are illustrated in the diagram below.

2.3 PrimaryKey Table vs Log Table Differences

2.4 Example Interpretation

The documentation's order table example clearly shows four changes and their representation in $changelog:

Note: An update operation is split into two independent records: update_before + update_after. This follows the standard CDC format (similar to Debezium). Downstream consumers use update_before to perform a retract (delete old value) and update_after to insert the new value.

2.5 Startup Mode (Consumption Starting Offset)

$changelog

supports three consumption startup modes, as illustrated:

2.6 Limitations

No Projection, Partition, or Predicate Pushdown support yet – currently SELECT * performs a full scan; partition pruning and predicate pushdown optimizations are not available. Even a query filtering on _change_type = 'insert' still scans all change records. Optimization is planned for future versions.

Binlog Virtual Table ( $binlog )

3.1 Use Cases

$binlog

provides complete before/after row images , closer to MySQL Binlog semantics, suitable for:

Data synchronization – downstream requires full before/after comparison.

Dual-write verification – compare data differences before and after updates.

Data replay – precisely restore data state at any point in time.

Applicability: Limited to PrimaryKey Tables; Log Tables do not support $binlog.

3.2 Schema Structure

The core difference from $changelog is that before and after are nested ROW types containing the full column structure of the original table.

3.3 Core Differences with $changelog

3.4 Example Interpretation

Comparing the user table example from the documentation:

$binlog

format is closer to MySQL or Debezium's before / after structure. For scenarios needing both before and after values (e.g., data comparison, conflict detection), $binlog carries the full context in a single record, which is more convenient to consume than $changelog 's paired records.

3.5 Nested Field Access

In Flink SQL, accessing fields inside before / after requires backtick escaping:

SELECT
    _change_type,
    `before`.name AS old_name,
    `after`.name AS new_name
FROM users$binlog
WHERE _change_type = 'update';

Design Philosophy Summary

4.1 Why Virtual Tables?

Traditional change log consumption typically requires:

Implementing complex parsing logic on the client side (identifying insert/update/delete semantics).

Using external tools (Debezium, Canal) to parse database binlogs.

Fluss exposes change logs directly as virtual tables, achieving:

SQL-native consumption – users consume change streams directly with Flink SQL, no extra tools needed.

Unified abstraction – both $changelog and $binlog are accessed via standard SQL interfaces.

Consistent permission model with regular tables – virtual tables reuse the base table's access control, no additional configuration required.

4.2 Internalizing Streaming Storage Capabilities

The core value of virtual tables lies in making "streaming data consumption capability" a native part of the database . Without extra components, Fluss completes change data capture (CDC), making it more efficient than traditional message queues (Kafka) or databases (MySQL) for building real-time data pipelines because it unifies storage, compute, and subscription in a single system.

4.3 $changelog vs $binlog Selection Guide