How Alibaba Cloud SLS Guarantees Zero‑Error Log Analysis with SQL Full‑Precision Mode

Background

When analyzing billions of log entries with SLS, some queries return “result not precise” because part of the data is not fully loaded and is excluded from SQL calculations.

Why Standard SQL Can Be Inaccurate

Data loading can be interrupted when time slices, data volume, row count or I/O operations exceed system thresholds. The limits are documented in the query and analysis restrictions.

Design Rationale

SLS isolates resources for instant and precise calculations, protecting multi‑tenant fairness and preventing malicious or accidental resource exhaustion.

SQL Full‑Precision Mode

This mode guarantees complete, accurate results by allocating dedicated resource pools and allowing the user to specify a maximum execution window. The query runs until it finishes or the window expires.

Implementation Principles

Resource Isolation: Separate pools for instant and precise calculations.

Time‑for‑Resources: Queries run until completion or timeout within the user‑defined window.

Load Splitting: Distinguish CPU‑intensive and I/O‑intensive tasks for optimal allocation.

Fine‑Grained Flow Control: Per‑user and per‑task QoS limits ensure stable service under high pressure.

When Full‑Precision Is Required

Business monitoring alerts where missed or false alarms are unacceptable.

Key‑metric operational analysis (revenue, conversion, retention) where small errors affect decisions.

Online data services needing 100 % accuracy for financial reconciliation or audit.

Complex multi‑column aggregations, wide‑column analysis (large JSON/text), and ultra‑large datasets (hundreds of GB to TB).

Optimization Recommendations

Limit the query time window to reduce scanned data.

Leverage indexes (full‑text, numeric, JSON) to accelerate scans, especially in full‑precision mode.

Pre‑process frequent complex queries with ScheduledSQL to create intermediate tables or views.

Validate on small data slices before scaling up.

Set an appropriate query_max_run_time to bound resource usage.

How to Enable

Console: Turn on “Full Precision” in query options.

Dashboard: Same option in the query panel.

API/SDK: Set session parameter, e.g., request.SetSession("allow_incomplete=false") in Java SDK.

// Java SDK example
public void demo() throws LogException {
    final String PROJECT = "...";
    final String LOGSTORE = "...";
    final String QUERY = "* | SELECT ...";
    final int FROM = (int)(System.currentTimeMillis()/1000) - 60;
    final int TO = (int)(System.currentTimeMillis()/1000);
    GetLogsRequest request = new GetLogsRequest(PROJECT, LOGSTORE, FROM, TO, "", QUERY);
    request.SetSession("allow_incomplete=false");
    GetLogsResponse response = client.GetLogs(request);
    System.out.println("Complete:" + response.IsCompleted());
}

Controlling Maximum Query Time

Set the session parameter query_max_run_time (e.g., set session query_max_run_time=100ms;) or via SDK ( GetLogsRequest.SetSession("query_max_run_time=100ms")). Exceeding the limit throws a LogException with HTTP 400.

Performance Comparison

Full‑precision mode delivers comparable performance to enhanced mode for most workloads; modest overhead appears only for ultra‑large datasets (hundreds of GB to TB).

SQL Mode Landscape

SLS provides three SQL modes: ordinary (fast, may be imprecise), enhanced (balanced), and full‑precision (guaranteed accuracy). Choose based on latency tolerance and precision needs.

Limitations

Memory per node limited to 10 GB; exceeding this causes query failure.

Synchronous queries timeout after 55 s; asynchronous queries after 10 min.

Concurrency per project limited to 5 queries (queue length 100).

Some internal errors may still mark results as imprecise.

Conclusion

SQL Full‑Precision mode trades additional execution time for guaranteed result completeness. It is suitable for critical monitoring, revenue and financial analysis, complex multi‑column aggregations, and ultra‑large log datasets where any inaccuracy could lead to incorrect business decisions.