0 views collected around this technical thread.
By switching from a naïve per‑record insert to JDBC batch writes with rewriteBatchedStatements and then off‑loading those batches to multiple consumer threads via a LMAX Disruptor ring buffer, the XML‑to‑MySQL import of 60,000 rows dropped from roughly 300 seconds to about 4 seconds while keeping memory usage modest.
Bilibili migrated its massive user‑behavior, commercial AI training, and database synchronization pipelines from Hive and Kafka to an Iceberg‑based streaming‑batch architecture, using Flink and the Magnus optimizer to achieve minute‑level freshness, reduce CPU and memory usage by about 20‑22 %, save roughly 3.55 M CNY annually, and dramatically improve query latency and join performance.
This guide demonstrates how to set up a SpringBoot environment, create a MySQL table, define a JavaBean, and perform efficient batch inserts using JdbcTemplate, BatchPreparedStatementSetter, NamedParameterJdbcTemplate, SimpleJdbcInsert, and MappingSqlQuery, with complete code examples for each technique.
This article outlines ten practical backend interface optimization techniques—including batch processing, asynchronous handling, caching, pre‑processing, pooling, parallelization, indexing, transaction management, code restructuring, and pagination—illustrated with code snippets and diagrams to help reduce latency and improve system efficiency.
This article introduces Flink Table Store v0.2, explains its four primary application scenarios—offline warehouse acceleration, partial update, pre‑aggregation rollup, and real‑time warehouse enhancement—details the core lake‑storage architecture, bucket management, append‑only mode, and outlines the project’s future roadmap and trade‑off considerations.
This article introduces the FlinkX framework for multi‑source heterogeneous data synchronization, detailing its background, core functions such as checkpoint‑based resume, metric monitoring, rate limiting, plugin architecture, cloud‑native K8s deployment, Hudi integration, and future roadmap, while also addressing common Q&A topics.
Apache Beam 2.28.0 introduces extensive Parquet support, new hash functions in BeamSQL and ZetaSQL, ApproximateDistinct via HLL, enhanced I/O connectors including SpannerIO for Numeric fields, ParquetIO schema support, KafkaTableProvider thrift, HadoopFormatIO key/value cloning skip, and various other improvements.
This article provides a comprehensive beginner-friendly guide to Windows batch (BAT) commands, covering common file operations, control structures, variable handling, and practical code examples for automating tasks on Windows virtual machines.