Building a Simple IM System with Alibaba Tablestore Timeline: A Step‑by‑Step Guide

Overview

Instant messaging applications such as DingTalk or WeChat rely on fast, reliable message storage and retrieval. Alibaba Tablestore’s Timeline model offers a natural fit for these scenarios. This guide demonstrates how to build a simple IM system using the Timeline (v1/v2) model, covering core features, data modeling, and implementation code.

Functional Modules

Message Storage

The storage repository holds messages for each conversation, identified by a unique timelineId. Messages are written either singly or in batches and read in reverse order within a specified sequence range. Table design example:

Key points:

Each conversation has its own TimelineQueue.

Messages are ordered by sequenceId.

Typical page size is 20‑30 messages.

Core code for fetching messages:

public List<AppMessage> fetchConversationMessage(String timelineId, long sequenceId) {
    TimelineStore store = timelineV2.getTimelineStoreTableInstance();
    TimelineIdentifier identifier = new TimelineIdentifier.Builder()
            .addField("timeline_id", timelineId)
            .build();
    ScanParameter parameter = new ScanParameter()
            .scanBackward(sequenceId)
            .maxCount(30);
    Iterator<TimelineEntry> iterator = store.createTimelineQueue(identifier).scan(parameter);
    List<AppMessage> appMessages = new LinkedList<>();
    int counter = 0;
    while (iterator.hasNext() && counter++ <= 30) {
        TimelineEntry entry = iterator.next();
        appMessages.add(new AppMessage(timelineId, entry));
    }
    return appMessages;
}

The storage repository must have a TTL of -1 to keep messages permanently.

Full‑Text and Multi‑Dimensional Search

To support fuzzy queries on message content, a multi‑field index is created on fields such as group ID, sender, message type, content (as a tokenized string), and timestamp.

public List<AppMessage> fetchConversationMessage(String timelineId, long sequenceId) {
    // Same implementation as above – retained for illustration of search‑enabled queries.
}

Sync Store (Real‑Time Notification)

The sync store records a lightweight copy of recent messages for fast unread‑count calculation. When a new message is written to the sync store, the system notifies online clients, which then pull messages after the stored checkpoint and update unread counters.

public List<AppMessage> fetchSyncMessage(String userId, long lastSequenceId) {
    TimelineStore sync = timelineV2.getTimelineSyncTableInstance();
    TimelineIdentifier identifier = new TimelineIdentifier.Builder()
            .addField("timeline_id", userId)
            .build();
    ScanParameter parameter = new ScanParameter()
            .scanForward(lastSequenceId)
            .maxCount(30);
    Iterator<TimelineEntry> iterator = sync.createTimelineQueue(identifier).scan(parameter);
    List<AppMessage> appMessages = new LinkedList<>();
    int counter = 0;
    while (iterator.hasNext() && counter++ <= 30) {
        appMessages.add(new AppMessage(userId, iterator.next()));
    }
    return appMessages;
}

Metadata Management

Two kinds of metadata are managed: user metadata (profile information) and conversation metadata (group name, type, creation time, etc.).

User metadata table example im_user_table:

Conversation metadata is stored via TimelineMeta and can be queried with multi‑field indexes for group name, tags, or geographic location.

Relationship Maintenance

Friendship and group membership are represented by relation tables.

Single‑chat relation table im_user_relation_table stores two rows per friendship (both directions) with the shared timeline_id:

public void establishFriendship(String userA, String userB, String timelineId) {
    PrimaryKey pkA = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("main_user", PrimaryKeyValue.fromString(userA))
            .addPrimaryKeyColumn("sub_user", PrimaryKeyValue.fromString(userB))
            .build();
    RowPutChange changeA = new RowPutChange(userRelationTable, pkA);
    changeA.addColumn("timeline_id", ColumnValue.fromString(timelineId));

    PrimaryKey pkB = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("main_user", PrimaryKeyValue.fromString(userB))
            .addPrimaryKeyColumn("sub_user", PrimaryKeyValue.fromString(userA))
            .build();
    RowPutChange changeB = new RowPutChange(userRelationTable, pkB);
    changeB.addColumn("timeline_id", ColumnValue.fromString(timelineId));

    BatchWriteRowRequest request = new BatchWriteRowRequest();
    request.addRowChange(changeA);
    request.addRowChange(changeB);
    syncClient.batchWriteRow(request);
}

public void breakupFriendship(String userA, String userB) {
    PrimaryKey pkA = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("main_user", PrimaryKeyValue.fromString(userA))
            .addPrimaryKeyColumn("sub_user", PrimaryKeyValue.fromString(userB))
            .build();
    RowDeleteChange delA = new RowDeleteChange(userRelationTable, pkA);

    PrimaryKey pkB = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("main_user", PrimaryKeyValue.fromString(userB))
            .addPrimaryKeyColumn("sub_user", PrimaryKeyValue.fromString(userA))
            .build();
    RowDeleteChange delB = new RowDeleteChange(userRelationTable, pkB);

    BatchWriteRowRequest request = new BatchWriteRowRequest();
    request.addRowChange(delA);
    request.addRowChange(delB);
    syncClient.batchWriteRow(request);
}

Group‑chat relation table im_group_relation_table maps group IDs to user IDs, enabling fast retrieval of all members via getRange. A secondary index im_group_relation_global_index supports reverse lookup (users → groups).

public List<Conversation> listMyGroupConversations(String userId) {
    PrimaryKey start = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("user_id", PrimaryKeyValue.fromString(userId))
            .addPrimaryKeyColumn("group_id", PrimaryKeyValue.INF_MIN)
            .build();
    PrimaryKey end = PrimaryKeyBuilder.createPrimaryKeyBuilder()
            .addPrimaryKeyColumn("user_id", PrimaryKeyValue.fromString(userId))
            .addPrimaryKeyColumn("group_id", PrimaryKeyValue.INF_MAX)
            .build();
    RangeRowQueryCriteria criteria = new RangeRowQueryCriteria(groupRelationGlobalIndex);
    criteria.setInclusiveStartPrimaryKey(start);
    criteria.setExclusiveEndPrimaryKey(end);
    criteria.setMaxVersions(1);
    criteria.setLimit(100);
    criteria.setDirection(Direction.FORWARD);
    criteria.addColumnsToGet(new String[]{"group_id"});
    GetRangeResponse resp = syncClient.getRange(new GetRangeRequest(criteria));
    List<Conversation> result = new ArrayList<>(resp.getRows().size());
    for (Row row : resp.getRows()) {
        String timelineId = row.getPrimaryKey().getPrimaryKeyColumn("group_id").getValue().asString();
        result.add(new Conversation(timelineId, describeGroup(timelineId)));
    }
    return result;
}

Instant Perception (Real‑Time Notification)

Instead of polling, a push‑based session pool records online client connections. When a new message is written, the pool notifies the corresponding client, which then pulls updates from the sync store. The session pool can be stored in memory or persisted in Tablestore.

Multi‑Device Synchronization

Checkpoint points (the latest read sequenceId) must be stored server‑side to keep unread counts consistent across devices. When a user reads a conversation, the server clears the unread counter and pushes the update to other online clients.

Sample Implementation

To run the sample, ensure the following:

Tablestore instance is created and accessible.

AccessKey (AK) and AccessKeySecret are obtained.

Configuration file tablestoreCong.json is placed in the home directory with fields endpoint, accessId, accessKey, and instanceName.

Secondary indexes are enabled on the instance.

{
  "endpoint": "http://instanceName.cn-hangzhou.ots.aliyuncs.com",
  "accessId": "***********",
  "accessKey": "***********************",
  "instanceName": "instanceName"
}

The project contains three entry points that must be executed in order, and the source code is open‑sourced on GitHub.