Jet Release Notes – New Features, Enhancements, Fixes, and Major Changes

Jet is an open‑source, in‑memory, distributed batch and stream processing engine capable of aggregating up to ten million events per second with sub‑10 ms latency.

New Features

[sql] Dynamic ClassDefinition eliminates the need to restart the cluster when using new record types in IMap.

Enhancements

[jdbc] Added an option to configure the batch size of the JDBC sink.

[sql] Optimized memory usage of SQL aggregations.

[sql] Expressions can now be used in SQL generator functions.

[core] Extracting data from IMap/ICache may cause OOME when too many partition iterators are created simultaneously.

[core] Removed misleading records of errors caused by cancelled jobs.

[pipeline‑api] Greatly improved performance of the pickAny aggregation operation in sliding windows.

Fixes

[core] Integration with Java Logging Framework unintentionally closed System.out from shutdown hooks, breaking other hooks.

[core] Fixed DAG.toDotString() to display the correct queue size.

[sql] CREATE MAPPING syntax bug allowing simultaneous use of OR REPLACE and IF NOT EXISTS in the same statement.

[cdc] Implemented processed offset feedback in CDC source.

[extensions] Updated AWS SDK to 1.11.976.

[extensions] Updated Guava to 30.1.

[extensions] Updated Parquet to 1.12.0.

[extensions] Updated Avro to 1.10.2.

[extensions] Updated Jetty to 9.4.38.v20210224.

[extensions] Updated wildfly‑openssl to 1.

[extensions] Updated ElasticSearch‑6 to 6.8.14.

[extensions] Updated ElasticSearch‑7 to 7.10.0.

[extensions] Updated Kafka to 2.2.2.

[extensions] Updated MySQL Connector to 8.0.20.

[extensions] Updated Apache Http Client to 4.5.13.

[extensions] Updated Netty to 4.1.61.Final.

[extensions] Updated Snakeyaml to 1.26 (SEC‑71).

Major Changes

The method signature of DAG.toDotString(int defaultParallelism) changed to DAG.toDotString(int defaultLocalParallelism, int defaultQueueSize) ; callers must now provide a queue size, which will be displayed when not overridden on the edge.