Mastering ZooKeeper Data Import/Export with MSE’s New Feature

Background

MSE offers a managed ZooKeeper service with higher SLA than self‑hosted open‑source ZooKeeper. At the end of 2022, MSE introduced a data import‑export feature that lets users handle ZooKeeper data without manual scripts.

Storage Principle

ZooKeeper persists two types of files:

Snapshot (snap) file : a full dump of the in‑memory DataTree representing the entire state of the cluster.

Transaction log (log) file : a sequential record of every client transaction applied to the in‑memory database.

Both files are required to fully restore a ZooKeeper cluster.

Snapshot generation is controlled by configurable parameters such as snapCount (default 100 000). The following code shows how a snapshot is written and when it is triggered:

public void save(DataTree dataTree, ConcurrentHashMap<Long, Integer> sessionsWithTimeouts, boolean syncSnap) throws IOException {
    long lastZxid = dataTree.lastProcessedZxid;
    File snapshotFile = new File(snapDir, Util.makeSnapshotName(lastZxid));
    try {
        snapLog.serialize(dataTree, sessionsWithTimeouts, snapshotFile, syncSnap);
    } catch (IOException e) {
        // handle exception
    }
}

private boolean shouldSnapshot() {
    int logCount = zks.getZKDatabase().getTxnCount();
    long logSize = zks.getZKDatabase().getTxnSize();
    return (logCount > (snapCount / 2 + randRoll))
           || (snapSizeInBytes > 0 && logSize > (snapSizeInBytes / 2 + randSize));
}

The snapshot stores a complete copy of the data tree, but it is not written in real time; it is created only when the conditions above are met. In the intervals between snapshots, new transactions are recorded only in the transaction log, providing the incremental changes.

Typical Use Cases

Data backup : Export the snapshot and log files to create a recoverable backup for disaster recovery or later restoration.

Cluster migration : Export data from an existing ZooKeeper cluster and import it into a new cluster, enabling a full data migration (requires a brief downtime).

Client behavior analysis : Parse exported snapshot and log files with ZooKeeper’s built‑in tools to identify abnormal client usage, such as oversized sessions or high‑frequency paths.

Hands‑On Practice

Below are the steps to export and import data using the MSE console (Professional edition only).

Exporting Data

Navigate to Data Management → Node Management → Data Export in the ZooKeeper Professional console.

Choose the desired file type: snapshot, transaction log, or both.

Click the Export button; the operation typically completes in 2–5 minutes.

Download the generated files from the download list.

Importing Data

Open Data Management → Node Management → Data Import .

Select the previously exported snapshot file (and optionally the log file).

Confirm the import; the system automatically initializes the target cluster with the imported state.

Conclusion

The MSE import‑export feature simplifies ZooKeeper data management, offering a fully automated backup, migration, and analysis workflow that is significantly faster than manual file handling, while ensuring cluster consistency by requiring both snapshot and transaction‑log files.