Implementing a Flutter Code Coverage Tool via Dill File Instrumentation

Background

Flutter lacks native code‑coverage tools, unlike Android (Jacoco) and iOS. To fill this gap, a custom coverage solution was built for the merchant‑app, which is heavily based on Flutter. The need arose because Flutter testing showed low efficiency, increased workload, reduced quality, and missing coverage metrics.

Principle

Line‑level coverage is achieved by recording whether each code block is executed. Instead of instrumenting every line (which would bloat the binary), the tool inserts a boolean flag at the start and end of each block. A simple Dart class example is shown, and the instrumented version records execution results.

Instrumentation Tool

The project uses Alibaba's open‑source AspectD , which modifies the compiled Dill (Dart Intermediate Language) files. AspectD adds an AOP step to the Flutter build pipeline, reads the Dill file, traverses all Dart AST nodes, inserts instrumentation code, and writes the modified Dill back.

Implementation Details

Key steps include converting Dill to an AST, creating a List<bool> coverage_insert array for each method, and inserting statements that set the corresponding flag to true at runtime. Example code for declaring the list and inserting statements is shown below:

///声明 List<bool> coverage_insert = CoverageImpl.getLibData(lib名字, 类名_方法名);
void createListDeclaration(Member procedure, Block node){
  VariableDeclaration declaration = VariableDeclaration(
      CoverageConstant.declarationListName,
      isFinal:true);
  declaration.type = InterfaceType(CoverageConstant.dartList.parent as Class,
      Nullability.legacy,
      [InterfaceType(CoverageConstant.dartBool, Nullability.legacy, null)]);
  Arguments arguments = Arguments.empty();
  String lib_key = library.importUri.toString();
  arguments.positional.add(StringLiteral(lib_key));
  String class_fun_key = getClassName(procedure) + CoverageConstant.class_fun_spile + procedure.name.name;
  arguments.positional.add(StringLiteral(class_fun_key));
  declaration.initializer = StaticInvocation(CoverageConstant.procedureGetListBool, arguments);
  declaration.parent = node;
  procedureListDeclaration = declaration;
  recordInsertInfo.insertStart(this, library, lib_key, class_fun_key);
}

///插入coverage_insert[procedureInsertTime] = true;
void insertStatement(Block node, {int position = 0})
{
  if(procedureListDeclaration == null){
    return;
  }
  Arguments arguments = Arguments.empty();
  Expression arg = IntLiteral(procedureInsertTime);
  arg.parent = arguments;
  arguments.positional.add(arg);
  arg = BoolLiteral(true);
  arg.parent = arguments;
  arguments.positional.add(arg);
  VariableGet get = VariableGet(procedureListDeclaration);
  MethodInvocation invocation = MethodInvocation(get,
      CoverageConstant.dartListDengYu.name, arguments,
      CoverageConstant.dartListDengYu);
  ExpressionStatement statement = ExpressionStatement(invocation);
  statement.parent = node;
  if(position >= node.statements.length){
    node.addStatement(statement);
  }else {
    node.statements.insert(position, statement);
  }
}

The tool also records insertion metadata (line number, block depth, method depth, type) for later analysis.

Data Collection and Diff

After instrumenting and running the app, the generated boolean arrays are exported as JSON. A diff of the current version against the previous one is obtained via git diff , and the changes are merged with coverage data to identify missed lines and methods.

{
  "package:example/test.dart": {
    "Test&**&": [true, true],
    "Test&**&getTestInfo": [true, true, true, false]
  }
}

Additional diff JSON snippets illustrate added/removed lines.

[
  {
    "add": false,
    "line": 8,
    "src": "      return \"无数据\";"
  },
  {
    "add": true,
    "line": 8
  }
]

Analysis, Statistics, and Reporting

Using the combined data, the tool computes line, method, and class coverage, as well as diff‑based metrics. An HTML report is generated, highlighting covered (green), uncovered (red), and unchanged code. Screenshots in the original article demonstrate overall coverage (~30%) and higher coverage on changed code (~70%).

Results and Impact

Since integration, the merchant‑app has seen reduced bug counts per test (from 4.3 to 3.0), shorter Flutter testing cycles (‑0.5 day), and a slight drop in error rate. The tool fills the previously empty Flutter coverage space.

Conclusion

The Flutter code‑coverage tool successfully instruments Dill files to provide line‑level coverage, though challenges remain such as environment setup, merging multiple test runs, version compatibility, and granularity compared to Jacoco. Future work will address these issues.