Log Management Practices for Reducing Cost and Improving Efficiency

Logs are the fastest-growing source of entropy in a system, carrying all side‑effects of business growth. This article presents a log‑governance case study that focuses on two themes: cost reduction and efficiency improvement.

The current situation suffers from three major kinds of chaos: classification chaos, format chaos, and tool chaos. Logs are split arbitrarily—by system module, tenant, ad‑hoc reasons—resulting in a proliferation of log files. Formats vary wildly, with some logs using overly complex delimiters or mixing stack traces with business messages. Different logging utilities further exacerbate inconsistency, as each tool emits its own structure.

To tame this, a three‑layer log hierarchy is proposed: Entry logs capture inbound/outbound parameters at traffic entry points (e.g., HSF, controller); Core logs record business‑level events during processing; Exit logs record parameters of third‑party service calls. Stack traces are treated as a separate layer because they have distinct characteristics.

Design guidelines include keeping classification simple and non‑overlapping, using ThreadLocal to store dimension data (industry, service, merchant, etc.), and adopting a fixed‑order, comma‑separated format. An example line looks like:

timeStamp|threadName logLevel loggerName|sourceAppName,flowId,traceId,sceneCode,identityCode,loginUserId,scpCode,rpcId,isYace,ip||businessCode,isSuccess||parameters||returnResult||

Stack traces are often folded in log viewers, hiding the root cause. The article suggests “reverse printing” of stack traces so that the deepest exception appears first, making the real error instantly visible. The core recursive algorithm is shown below.

/**
 * Recursively reverse‑print stack trace and its causes (starting from the deepest exception).
 * @param t original throwable
 * @param causeDepth max depth of cause recursion
 * @param counter depth counter (must be mutable across frames)
 * @param stackDepth max depth of each stack trace
 * @param sb StringBuilder for output
 */
public static void recursiveReversePrintStackCause(Throwable t, int causeDepth, ForwardCounter counter, int stackDepth, StringBuilder sb){
    if(t == null){
        return;
    }
    if (t.getCause() != null){
        recursiveReversePrintStackCause(t.getCause(), causeDepth, counter, stackDepth, sb);
    }
    if(counter.i++ < causeDepth){
        doPrintStack(t, stackDepth, sb);
    }
}

The helper that prints a limited number of stack frames while shortening class names is:

public static void doPrintStack(Throwable t, int stackDepth, StringBuilder sb){
    StackTraceElement[] stackTraceElements = t.getStackTrace();
    if(sb.lastIndexOf("\t") > -1){
        sb.deleteCharAt(sb.length()-1);
        sb.append("Caused: ");
    }
    sb.append(t.getClass().getName()).append(": ").append(t.getMessage()).append("\n\t");
    for(int i=0; i < stackDepth; ++i){
        if(i >= stackTraceElements.length){
            break;
        }
        StackTraceElement element = stackTraceElements[i];
        sb.append(reduceClassName(element.getClassName()))
          .append("#")
          .append(element.getMethodName())
          .append(":")
          .append(element.getLineNumber())
          .append("\n\t");
    }
}

Applying these techniques can compress log size by up to 88% while preserving essential information. The article concludes that while the three‑layer approach works well for typical middle‑platform services, adaptations may be needed for client‑heavy applications, and that no single “silver bullet” fits every scenario.