StarRocks Production Troubleshooting: Data Loss, Memory Spikes, and Global Dictionary Errors

Issue 1: Data Loss During MaxCompute to StarRocks Integration

Data was processed in MaxCompute to the ADS layer and then loaded into StarRocks for analysis. The integration task logs showed no errors, but a comparison of distinct column values between MaxCompute and StarRocks revealed that longer string values present in MaxCompute were missing in StarRocks.

Investigation showed the StarRocks table column was defined as varchar(50), while the MaxCompute source column was a string type with an 8 MB length limit. The length mismatch caused silent truncation.

Solution:

Alter the StarRocks column to a larger length:

ALTER TABLE table_name MODIFY COLUMN column_name varchar(500);

(Note: the original article shows a typo "ALTRER"; the correct keyword is ALTER.)

Re-run the integration task and verify data integrity.

Issue 2: BE Node Memory Spike Due to Timeout Mismatch

The API gateway timeout was set to 30 seconds, while StarRocks query timeout was 300 seconds. When resources were constrained, queries initiated via the gateway could not return within 30 seconds and timed out. The application's retry logic (try-catch) then re-issued the same query, causing multiple concurrent query tasks to run in the cluster and driving up BE memory usage.

Optimization: Set the query timeout at the SQL level to match the gateway timeout (30 seconds) using a session variable hint:

select /*+ SET_VAR(query_timeout=30) */ ...

Issue 3: Global Dictionary Error "Not Found string in global dict: 2024-11"

A report query failed with the error "Not Found string in global dict: 2024-11". This was caused by the low-cardinality global dictionary optimization incorrectly handling a value.

Fix: Disable the low-cardinality global dictionary optimization by setting the session variable:

cbo_enable_low_cardinality_optimize=false