Design and Implementation of MetrAutoAPI: A Flexible Metric Automation Platform

Background – The company is undergoing a data‑strategy transformation, facing increasing data demands and richer metric data. Traditional data‑business development processes are inefficient and cannot quickly respond to changing requirements, prompting the need for a fast, flexible solution.

2. MetrAutoAPI Design

2.1 Platform Introduction – MetrAutoAPI (Metric Automate API) decouples metric data from the application layer, providing a unified API for all data requests and responses. It integrates multiple data sources, offers configurable API modeling via drag‑and‑drop UI, and includes a rule‑engine service for dynamic calculations.

2.2 Architecture Design – The architecture consists of four layers: a physical query layer (unified query engine for heterogeneous databases), a semantic model layer (metric metadata and API‑SQL model management), a unified service layer (API construction, data query, and rule‑engine configuration), and a unified interface layer (single external API endpoint). Diagrams illustrate the layered structure and its advantages over traditional development.

2.3 Usage Scenarios – Examples include data‑dashboard display and other metric‑driven applications.

2.4 Core Functions

2.4.1 SQL Modeling Service – Utilizes Zealot and MySqlStatementParser to generate SQL statements dynamically based on user‑defined conditions, avoiding manual SQL concatenation. The following Java method extracts field names from a SQL string:

public static List<FieldVo> getFieldName(String sqlStr) {
    MySqlStatementParser mySqlStatementParser = new MySqlStatementParser(sqlStr);
    // parse AST
    SQLStatement sqlStatement = mySqlStatementParser.parseStatement();
    MySqlSchemaStatVisitor visitor = new MySqlSchemaStatVisitor();
    sqlStatement.accept(visitor);
    List<FieldVo> list = new ArrayList<>();
    Collection<TableStat.Column> columns = visitor.getColumns();
    columns.stream().forEach(row -> {
        if (row.isSelect()) {
            FieldVo fieldVo = new FieldVo();
            fieldVo.setTableName(row.getTable());
            fieldVo.setFieldName(row.getName());
            list.add(fieldVo);
        }
    });
    List<FieldVo> aliasList = getAliasField(sqlStr);
    for (int i = 0; i < list.size(); i++) {
        FieldVo vo = list.get(i);
        FieldVo aliasVo = aliasList.get(i);
        if (Objects.isNull(aliasVo.getAliasName())) {
            vo.setAliasName(vo.getFieldName());
        } else {
            vo.setAliasName(aliasVo.getAliasName());
        }
    }
    return list;
}

2.4.2 API‑Aviator Rule Engine Service – Aviator, a high‑performance Java expression evaluator, separates complex business rules from application code. Custom functions (e.g., TransNullToZeroRule, IsNullRuleFunction) are registered to support dynamic rule execution.

public class AviatorEvaluatorUtils {
    private static AviatorEvaluatorInstance instance = AviatorEvaluator.getInstance();
    static {
        instance.addFunction(new TransNullToZeroRule());
        instance.addFunction(new IsNullRuleFunction());
        instance.addFunction(new CrrRadioRule());
        instance.addFunction(new CrrRule());
    }
    public static AviatorEvaluatorInstance getInstance() { return instance; }
}

The engine compiles and executes expressions for each metric field, handling formatting, type conversion, and error logging.

private List<Map<String, Object>> expResult(List<RestApiVo> fieldList, List<Map<String, Object>> dataList) {
    Stopwatch started = Stopwatch.createStarted();
    List<Map<String, Object>> resultList = new ArrayList<>();
    for (Map<String, Object> map : dataList) {
        Map<String, Object> dataMap = new HashMap<>();
        fieldList.forEach(e -> {
            try {
                Expression exp = AviatorEvaluatorUtils.getInstance().compile(e.getFieldExp(), true);
                Object value = exp.execute(map);
                if (Objects.isNull(value)) {
                    dataMap.put(e.getFieldName(), null);
                } else {
                    if (e.getIsFormat().intValue() == 1) {
                        BigDecimal decimal = new BigDecimal(value.toString());
                        BigDecimal scale = decimal.setScale(e.getNumberFormat().intValue(), BigDecimal.ROUND_HALF_UP);
                        dataMap.put(e.getFieldName(), scale);
                    } else {
                        dataMap.put(e.getFieldName(), value);
                    }
                }
            } catch (Exception ex) {
                log.error("解析表达式异常,字段:{},结果:{}", JsonUtil.serialize(e), JsonUtil.serialize(map), ex);
                dataMap.put(e.getFieldName(), null);
            }
        });
        resultList.add(dataMap);
    }
    log.warn("转换结果耗时:{}", started.stop());
    return resultList;
}

2.4.3 Unified Query Engine – Provides a standardized API endpoint (/restApi) that validates parameters, invokes the underlying service, and returns results in a consistent protocol. The controller includes comprehensive validation of appId, apiKey, and request payload.

@PostMapping("/restApi")
public Protocol<List<Map<String, Object>>> restApi(String _appId, @RequestBody Map<String, Object> params, String apiKey) {
    ParamsValid valid = new ParamsValid();
    valid.validNotNull("params", params)
         .valid("apiId与apiKey不能同时为空", () -> {
             if (Objects.isNull(params.get("apiId")) && Objects.isNull(params.get("apiKey"))) {
                 return false;
             }
             return true;
         })
         .valid("apiKey值不正确", () -> {
             if (Objects.nonNull(params.get("apiKey"))) {
                 if (Strings.isNullOrEmpty(params.get("apiKey").toString())) {
                     return false;
                 }
             }
             return true;
         });
    if (!valid.isValid()) {
        return valid.showInValidMessage();
    }
    Protocol<List<Map<String, Object>>> protocol = null;
    try {
        protocol = targetAutoService.restApi(params, _appId, 1);
    } catch (DataSelfException ex) {
        log.error("查询数据异常:param:{},apiKey:{}", JsonUtil.serialize(params), apiKey, ex);
        return new Protocol<>(-1, ex.toString());
    } catch (Exception e) {
        log.error("查询信息异常:param:{}", JsonUtil.serialize(params), e);
        return new Protocol<>(-1, "查询异常，请重试");
    }
    return protocol;
}

3. Practice Issues and Solutions

3.1 Performance Bottleneck – Database reads for configuration and rule metadata caused latency. The solution was to cache metadata in Redis, falling back to the database when the cache missed, dramatically reducing response time.

3.2 High Deployment Cost – Separate modeling in test and production environments led to inconsistencies. The team introduced a JSON‑based metadata export/import mechanism to replicate models across environments safely.

3.3 Multiple API Aggregation – Clients often needed many metric APIs, increasing request volume. An API aggregation layer was added to combine multiple metric responses into a single endpoint, improving concurrency handling and reducing latency.

4. References

ApiJson: http://apijson.cn/doc/zh/

Mybatis Dynamic SQL: https://github.com/mybatis/mybatis-dynamic-sql

Zealot: https://gitee.com/chenjiayin1990/zealot

Aviator: https://www.yuque.com/boyan-avfmj/aviatorscript