Tagged articles
checkpointing
33 articles · Page 1 of 1
Google's Agent Substrate 1.0 separates stateful Actors from reusable Worker pools, enabling AI agents to checkpoint memory and filesystem state, release compute during idle waits, and resume in under 500 milliseconds on GKE, achieving 10x density over standard containers while preserving isolation via gVisor or microVMs.
AI agentsAgent SubstrateGKE
0 likes · 15 min read

This article explains why treating hour-long tasks as extended short requests fails, detailing required patterns: bounded waiting with heartbeats to prevent connection drops, two-level timeouts (per-step and total), partial failure semantics for batch jobs, explicit artifact ownership to avoid cross-session contamination, and acknowledges the critical lack of checkpointing/resume, advising task decomposition instead.
agent platformartifact ownershipbounded waiting
0 likes · 21 min read

This chapter distinguishes scheduling from batch execution, covering single-machine, cluster, distributed, and pipeline batch patterns, emphasizing reliability through state tracking, idempotency, sharding, checkpoints, and explicit dependencies over fixed timing. It also outlines practical platform capabilities like versioned task definitions, unique run instances, state machines, leases, business idempotency keys, resource quotas, and end-to-end reconciliation.
IdempotencyPipelineScheduling
0 likes · 9 min read


TonyBaiOct 3, 2026 · Artificial Intelligence
The article analyzes Pi 1.0 and Pi Durable, a minimalist agent harness framework that now supports MCP via Codemode, introduces virtual model routing, and provides six durability primitives including crash recovery, session forking, and background compression for long-running, multi-user agent applications.
Agent HarnessCodemodeCrash Recovery
0 likes · 27 min read


ArchitectSep 20, 2026 · Artificial Intelligence
This article explains the Workflow architecture for multi-agent systems where code controls routing over fixed paths while agents handle node-internal reasoning, detailing checkpoint placement at node boundaries for retry/pause/recovery, required node output fields, event logging, and a comparison of how LangGraph, CrewAI, Microsoft Agent Framework, OpenAI Agents SDK, and Claude Agent SDK express such workflows.
Agent OrchestrationCrewAILangGraph
0 likes · 20 min read


Qborfy AIAug 31, 2026 · Artificial Intelligence
This article provides a step‑by‑step walkthrough for constructing a daily industry research‑report automation system using LangGraph, covering fan‑out parallel search, fan‑in aggregation, an evaluator‑optimizer feedback loop, human‑in‑the‑loop approval, checkpointing for resumability, and full observability with Langfuse, complete with runnable code.
AI automationHITLLangGraph
0 likes · 24 min read


Qborfy AIAug 29, 2026 · Artificial Intelligence
This article examines common pitfalls in State design for LangGraph AI workflows, explains how Reducer functions resolve concurrent writes, compares short‑term, thread‑level, and long‑term memory architectures, and demonstrates practical Checkpointing and Time‑Travel techniques for robust session persistence.
AI agentsLangGraphReducer
0 likes · 21 min read

In large‑model training, a smooth loss decline is only a necessary condition; the article explains which additional logs—learning‑rate, gradient norm, parameter updates, validation metrics, throughput, GPU usage, and checkpoint status—must be monitored and how to diagnose common issues with concrete code examples.
GPU utilizationPyTorchcheckpointing
0 likes · 22 min read

The article outlines five concrete design patterns—Checkpoint‑and‑Resume, Delegated Approval, Memory‑Layered Context, Ambient Processing, and Fleet Orchestration—that enable production‑grade, multi‑day AI agents to persist state, handle failures, and scale safely.
AI agentscheckpointingcloud sandbox
0 likes · 12 min read

When a scheduled‑task server crashes, simply restarting it is insufficient; a robust solution must combine clustering, distributed locks, idempotent designs, checkpointing, and monitoring to ensure tasks resume correctly across non‑runtime and runtime failures, as detailed with SpringTask‑Redis and XXL‑JOB implementations.
BackendDistributed LockIdempotency
0 likes · 28 min read

Alluxio’s distributed caching dramatically accelerates AI training and checkpointing workloads, achieving up to 99.57% GPU utilization and linear scaling across clusters in the MLPerf Storage v2.0 benchmark, while using cost‑effective commodity hardware to eliminate I/O bottlenecks.
AI TrainingAlluxioDistributed Caching
0 likes · 11 min read

The article traces the evolution of AI training stability from early manual operations on small GPU clusters to sophisticated, fault‑tolerant infrastructures for thousand‑card and ten‑thousand‑card models, detailing Baidu Baige’s metrics, monitoring, eBPF‑based diagnostics, and checkpoint strategies that reduce invalid training time and accelerate fault recovery.
checkpointingdistributed systemseBPF
0 likes · 22 min read

The article examines how Baidu Baige evolved AI training stability from manual operations to precise engineering, detailing metrics, fault‑perception techniques, eBPF‑based diagnostics, multi‑level restart strategies, and trigger‑based checkpointing that together achieve sub‑minute recovery and 99.5% effective training time on massive GPU clusters.
AI TrainingLarge-Scale Clusterscheckpointing
0 likes · 25 min read

The article provides a comprehensive English overview of 3FS, an AI‑focused distributed file system that leverages FoundationDB for metadata, CRAQ for chunk replication, and a hybrid Fuse/native client architecture, detailing its design, components, fault handling, and performance considerations for large‑scale training workloads.
AI storageCRAQ replicationFoundationDB
0 likes · 25 min read

This article explains what Flink’s Exactly‑Once semantics actually guarantee, why it does not mean each event is processed only once, how checkpointing and two‑phase commit sinks enable end‑to‑end exactly‑once, and the three safeguards needed for true exactly‑once computation.
Flinkbig datacheckpointing
0 likes · 5 min read

Researchers from Kwai’s large-model team present a novel training system that combines pipeline-parallel-aware activation offloading with a compute-memory balanced checkpointing strategy, enabling lossless acceleration of large language models, achieving up to 42.7% MFU on 256 NVIDIA H800 GPUs while reducing memory usage.
GPU trainingKwaiPerformance Modeling
0 likes · 13 min read

This article explains how GPU memory is allocated during model fine‑tuning, describes collective communication primitives, and compares data parallel, model parallel, ZeRO, pipeline parallel, mixed‑precision, and checkpointing techniques for reducing memory consumption in large‑scale AI training.
GPU memoryPipeline ParallelZeRO
0 likes · 9 min read

The article reviews Flink 1.19’s new features, highlighting SQL capability enhancements such as custom source parallelism, TTL hints, and MiniBatch support for regular joins, as well as runtime dynamic parallelism for batch jobs and flexible checkpointing intervals for different data sources.
FlinkParallelismSQL
0 likes · 6 min read

This article shares ten practical pitfalls encountered when moving hourly Spark session jobs to Flink, covering parallelism load imbalance, state TTL, checkpointing strategies, logging, JMX debugging, state migration risks, reduce vs process choices, input data validation, event‑time handling, and external storage considerations, along with concrete configuration snippets and performance tips.
FlinkPerformance TuningSpark migration
0 likes · 20 min read

This article shares ten practical pitfalls encountered while moving hour‑level Spark session processing jobs to Apache Flink, covering parallelism skew, state TTL, checkpoint handling, logging, debugging, state migration, Reduce vs Process, input validation, event‑time handling, and the trade‑offs of storing data inside Flink.
FlinkState Managementbig data
0 likes · 19 min read

Apache Flink 1.11.0 introduces a suite of major enhancements—including unaligned checkpoints, a unified source interface, CDC support in Table API/SQL, performance‑boosted PyFlink, a new application deployment mode, and numerous UI, Docker, and catalog improvements—aimed at increasing usability, scalability, and integration across streaming and batch workloads.
FlinkSQLSource Interface
0 likes · 18 min read

This article explains Apache Flink's distributed system architecture—including JobManager, ResourceManager, TaskManager, and Dispatcher—covers session and job deployment modes, data transfer mechanisms, event‑time handling with watermarks, various state types and backends, scaling strategies, and the checkpoint/savepoint recovery process.
Apache FlinkEvent TimeState Management
0 likes · 15 min read

This article provides a detailed introduction to Spark Streaming, covering its architecture, DStream concepts, initialization, data sources, transformations, windowed aggregations, output operations, checkpointing, fault‑tolerance semantics, deployment, performance tuning, and monitoring for building reliable high‑throughput streaming applications.
DstreamScalaSpark
0 likes · 17 min read

This article explains Flink's StreamingFileSink in version 1.10.0, covering how files transition through In‑progress, Pending, and Finished states, the bucket assignment and rolling policies, and provides a complete Java example for writing string data to files.
File RollingFlinkJava
0 likes · 6 min read

This article provides a comprehensive overview of Apache Flink, explaining how it unifies stream and batch processing on a single runtime, detailing its key features, APIs, libraries, architectural components, fault‑tolerance mechanisms, scheduling, iterative processing, and back‑pressure monitoring.
Apache Flinkbackpressurebatch processing
0 likes · 20 min read

This article explains how Apache Flink implements fault‑tolerant checkpointing using the Asynchronous Barrier Snapshot (ABS) algorithm, a localized version of the Chandy‑Lamport distributed snapshot, covering barriers, snapshot alignment, exactly‑once versus at‑least‑once semantics, and handling of cyclic dataflow graphs.
Asynchronous Barrier SnapshotFlinkcheckpointing
0 likes · 9 min read

This article explains how to use Flink 1.9's built‑in Kafka connector, covering supported versions, Maven dependencies, consumer and producer configuration in Java and Scala, checkpointing, offset handling, partition discovery, timestamps, watermarks, and provides a complete runnable example.
ConnectorFlinkJava
0 likes · 12 min read

This article demonstrates how to develop a real‑time ETL job using Apache Flink, covering project setup, Kafka as a source, custom bucket assigners for HDFS, checkpointing, savepoints, and deployment on YARN to achieve exactly‑once processing guarantees.
Apache FlinkKafkaStreamingFileSink
0 likes · 11 min read

This article explores the two toughest problems in distributed stream processing—exactly‑once message handling and ordering—by dissecting the underlying impossibility of perfect failure detectors, the liveness‑vs‑safety trade‑off, zombie processes, and the practical solutions employed by systems such as Flink, Kafka Streams, MillWheel, and Spark.
Consensuscheckpointingdistributed systems
0 likes · 81 min read

This article explains Spark Streaming’s architecture, core concepts such as DStream, windowing, and the two Kafka integration methods—Receiver-based and Direct approaches—detailing their configurations, memory implications, checkpointing, and best‑practice recommendations for reliable, high‑throughput real‑time data processing.
Direct ApproachReceiver ApproachSpark Streaming
0 likes · 18 min read

This article explains Apache Flink's fault‑tolerance mechanisms, including checkpointing, barrier alignment, the differences between At‑Least‑Once and Exactly‑Once semantics, configuration options, incremental checkpointing, and the requirements for external sources and sinks to achieve end‑to‑end exactly‑once processing.
Apache Flinkbig datacheckpointing
0 likes · 15 min read

This tutorial demonstrates how to use Apache Flink's Kafka connector to read data from Kafka topics with exactly‑once semantics, covering Maven dependencies, consumer configuration, checkpointing for fault tolerance, and a complete Scala example that writes the streamed data to HDFS.
FlinkKafkaConnectorScala
0 likes · 5 min read

This article explains the core concepts of Paxos, its role in asynchronous distributed environments, and the practical engineering techniques used to create a production‑ready Paxos library, covering roles, instance management, optimization, checkpointing, and correctness guarantees.
Distributed ConsensusPaxosProduction-Grade
0 likes · 28 min read
