Understanding OceanBase Write Throttling Mechanism and Configuration

1. Introduction

OceanBase includes a write throttling mechanism to control the rate of write operations (insert, update, delete) in order to improve database stability. The mechanism is governed by two parameters: writing_throttling_trigger_percentage (default 60, range 1‑100, 100 disables throttling) and writing_throttling_maximum_duration (default 2 hours).

Note: This feature is supported only from OceanBase 2.2.30 onward.

2. Implementation Principle

1. Entering Throttling Logic

When a write operation requests memory, throttling is triggered if the used Memstore memory exceeds the percentage defined by writing_throttling_trigger_percentage.

2. Multiple Throttling Passes

The throttling logic splits the current memory request into several passes, each lasting at most 20 ms.

3. Throttling Loop Conditions

Memory release: enough memory has been freed, so throttling can stop.

SQL execution time limit: the SQL has reached its time limit, ending the loop.

Sleep time: the loop has slept for 20 seconds, then exits.

4. Completing Throttling

If any of the above conditions is met, the system exits the throttling loop and returns the SQL completion information to the client.

5. Process Flow Diagram

3. Source Code Walkthrough

The following code snippets illustrate how the two parameters affect the throttling logic during an INSERT operation.

int ObTablet::insert_row_without_rowkey_check(
    ObRelativeTable &relative_table,
    ObStoreCtx &store_ctx,
    const common::ObIArray<share::schema::ObColDesc> &col_descs,
    const storage::ObStoreRow &row)
{<br/>  int ret = OB_SUCCESS;<br/>  {<br/>    // insert_row_without_rowkey_check executes and then the destructor of ObStorageTableGuard performs throttling<br/>    ObStorageTableGuard guard(this, store_ctx, true);<br/>    ObMemtable *write_memtable = nullptr;<br/>    ...<br/>    // write_memtable‑>set() triggers ObFifoArena::alloc() which performs throttling check<br/>    else if (OB_FAIL(write_memtable‑>set(store_ctx, relative_table.get_table_id(),<br/>    full_read_info_, col_descs, row)))<br/>    ...<br/>  }<br/>  return ret;<br/>}

The throttling logic resides in ObFifoArena::alloc() and its helper ObFifoArena::speed_limit():

void* ObFifoArena::alloc(int64_t adv_idx, Handle& handle, int64_t size)
{<br/>  int ret = OB_SUCCESS;<br/>  void* ptr = NULL;<br/>  int64_t rsize = size + sizeof(Page) + sizeof(Ref);<br/>  // call speed_limit to decide throttling<br/>  speed_limit(ATOMIC_LOAD(&hold_), size);<br/>  ...<br/>}

The speed_limit method checks the current memory usage against the trigger percentage, calculates a decay factor based on writing_throttling_maximum_duration, and sets a thread‑local flag indicating whether throttling is needed.

void ObFifoArena::speed_limit(const int64_t cur_mem_hold, const int64_t alloc_size)
{<br/>  int64_t trigger_percentage = get_writing_throttling_trigger_percentage_();<br/>  if (trigger_percentage < 100) {<br/>    if (cur_mem_hold > lastest_memstore_threshold_ * trigger_percentage / 100) {<br/>        need_speed_limit = true;<br/>        int64_t alloc_duration = get_writing_throttling_maximum_duration_();<br/>        throttle_info_.check_and_calc_decay_factor(...);<br/>    }<br/>  }<br/>  // record logs, set thread‑local variables, etc.
}

The actual throttling (sleep) is performed in the destructor of ObStorageTableGuard when the write operation finishes:

ObStorageTableGuard::~ObStorageTableGuard()
{<br/>  if (need_control_mem_ && tl_need_speed_limit()) {<br/>    while (need_sleep && ... ) {<br/>        int64_t sleep_interval = min(min(left_interval, SLEEP_INTERVAL_PER_TIME), expected_wait_time);<br/>        ::usleep(sleep_interval);<br/>        ...<br/>    }<br/>  }<br/>  reset();<br/>}

4. Usage Instructions

The parameters are tenant‑level settings and can be modified via SQL:

obclient> ALTER SYSTEM SET writing_throttling_trigger_percentage = 80;
obclient> ALTER SYSTEM SET writing_throttling_maximum_duration = '2h';
-- or specify a tenant
obclient> ALTER SYSTEM SET writing_throttling_trigger_percentage = 80 tenant=<tenant_name>;

5. Application Scenarios

When creating a tenant with heavy write load (e.g., bulk data import), enabling throttling prevents Memstore exhaustion.

If QPS drops unexpectedly during write‑intensive workloads, checking the memstore memory page alloc wait event in v$session_event or the logs can confirm whether throttling is the cause.

Typical diagnostic queries:

SELECT * FROM v$session_event WHERE EVENT='memstore memory page alloc wait' \G;

Relevant log patterns:

[INFO] speed_limit ... report write throttle info(...)
[INFO] ~ObStorageTableGuard ... throttle situation(...)

6. Summary

The write throttling feature in OceanBase is implemented in the destructor of ObStorageTableGuard, which runs after a memtable write completes. This post‑write throttling delays subsequent memory allocations without affecting the current write, thereby controlling memory consumption and maintaining system stability.