How to Safely Delete Standby Tenants in OceanBase OBD 3.6.0

1. Background

When using OBD 3.6.0 to delete a standby tenant, the command actually removes the primary tenant and keeps the standby tenant, which is contrary to expectations.

Convention : All cluster names refer to the deploy_name specified during OBD installation. Variables are denoted as {xxx}.

2. Environment Preparation

Prepare a pair of primary‑standby tenants.

Function : Show information of a specified tenant in a given cluster.

Syntax : obd cluster tenant show {cluster_name} -t {tenant_name} -g Parameters : -t tenant name, -g display primary‑standby topology graph.

# obd cluster tenant show my_ob_master -t tnt_mysql -g
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                            tenant basic info                                            |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tnt_mysql   | USER        | MYSQL              | RANDOM       | 2.0     | 2.0     | 2G          | 10000    | 10000    | 4G            | 2           | PRIMARY     |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
query primary-standby relation ok

primary-standby relation topology graph

my_ob_master:tnt_mysql
└── my_ob1:tnt_mysql

Primary tenant cluster : my_ob_master:tnt_mysql Standby tenant cluster :

my_ob1:tnt_mysql

3. Test Process

3.1 View standby tenant status from the standby side

# obd cluster tenant show my_ob1 -t tnt_mysql -g
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                            tenant basic info                                            |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tnt_mysql   | USER        | MYSQL              | RANDOM       | 2.0     | 2.0     | 2G          | 10000    | 10000    | 4G            | 2           | STANDBY    |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                    standby tenant standby info                    |
+---------------------+---------------+-------------+----------------------------+----------+-------------------+-------------------+------------------+----------+
| standby_tenant_name | tenant_status | sync_status | sync_scn_timestamp         | err_code | error_comment    | switchover_status | switchover_epoch | log_mode |
+---------------------+---------------+-------------+----------------------------+----------+-------------------+-------------------+------------------+----------+
| tnt_mysql           | NORMAL        | NORMAL      | 2025-11-16 22:30:14.391364 | 0        |                   | NORMAL            | 0                | NOARCHIVELOG |
+---------------------+---------------+-------------+----------------------------+----------+-------------------+-------------------+------------------+----------+
query primary-standby relation ok

primary-standby relation topology graph

my_ob_master:tnt_mysql
└── my_ob1:tnt_mysql

The standby tenant standby info shows tenant_status and sync_status are both NORMAL, indicating the standby tenant is healthy.

3.2 View primary tenant status from the primary side

# obd cluster tenant show my_ob_master -t tnt_mysql -g
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                            tenant basic info                                            |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tnt_mysql   | USER        | MYSQL              | RANDOM       | 2.0     | 2.0     | 2G          | 10000    | 10000    | 4G            | 2           | PRIMARY     |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
query primary-standby relation ok

primary-standby relation topology graph

my_ob_master:tnt_mysql
└── my_ob1:tnt_mysql

3.3 Delete the standby tenant

Function : Delete a specific tenant in a cluster.

Syntax : obd cluster tenant drop {cluster_name} -t {tenant_name} Parameter : -t tenant to delete.

# obd cluster tenant drop my_ob1 -t tnt_mysql
Get local repositories ok
Open ssh connection ok
Connect to observer 10.186.65.22:2881 ok
Get deployment connections ok
Get standbys info ok
Drop tenant tnt_mysql ok
Trace ID: cc90eeba-c2f8-11f0-9a4c-02000aba4116
If you want to view detailed obd logs, please run: obd display-trace cc90eeba-c2f8-11f0-9a4c-02000aba4116

3.4 Verify deletion from the standby side

# obd cluster tenant show my_ob1 -t tnt_mysql -g
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                            tenant basic info                                            |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
| tnt_mysql   | USER        | MYSQL              | RANDOM       | 2.0     | 2.0     | 2G          | 10000    | 10000    | 4G            | 2           | STANDBY    |
+-------------+-------------+--------------------+--------------+---------+---------+-------------+----------+----------+---------------+-------------+-------------+
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                    standby tenant standby info                    |
+---------------------+---------------+-------------------+----------------------------+----------+-------------------------------------------+-------------------+------------------+----------+
| standby_tenant_name | tenant_status | sync_status       | sync_scn_timestamp         | err_code | error_comment                            | switchover_status | switchover_epoch | log_mode |
+---------------------+---------------+-------------------+----------------------------+----------+-------------------------------------------+-------------------+------------------+----------+
| tnt_mysql           | NORMAL        | SOURCE HAS A GAP  | 2025-11-16 22:30:32.923425 | -4233    | There is a gap between the log source ...| NORMAL            | 0                | NOARCHIVELOG |
+---------------------+---------------+-------------------+----------------------------+----------+-------------------------------------------+-------------------+------------------+----------+
query primary-standby relation ok

The tenant basic info still contains tnt_mysql, meaning the standby tenant was not removed.

3.5 Verify deletion from the primary side

# obd cluster tenant show my_ob_master -t tnt_mysql -g
[ERROR] tnt_mysql not exists in my_ob_master

The primary tenant has been deleted.

4. How to Safely Delete a Standby Tenant

4.1 Decouple the standby tenant from its primary

After decoupling, the standby tenant becomes an independent primary tenant.

Function : Break the primary‑standby relationship for a standby tenant.

Syntax :

obd cluster tenant decouple {standby_cluster} {standby_tenant} -p {root_password}

Parameter : -p root password of the standby tenant.

# obd cluster tenant decouple my_ob1 tnt_mysql -p aaAA11__

4.2 Delete the original standby tenant after decoupling

# obd cluster tenant drop my_ob1 -t tnt_mysql

5. Additional Scenarios

If the primary tenant is accidentally deleted but the standby tenant remains, a failover can promote the standby tenant to primary:

Function : Promote standby tenant via failover.

Syntax :

obd cluster tenant failover {standby_cluster} {standby_tenant} -p {root_password}

Parameter : -p root password.

# obd cluster tenant failover my_ob1 tnt_mysql -p aaAA11

Conclusion

OBD 3.6.0 deletes the primary tenant when attempting to delete a standby tenant, which is unexpected.

To delete a standby tenant while keeping the primary, first decouple the standby tenant, then drop the original standby tenant.

If the primary tenant is lost, try to recover it from a recycle bin; if not possible, use failover to promote the standby tenant and then reconcile missing data.