Boosting Developer Experience: A Deep Dive into Documentation Engineering

Pain Points of Documentation Engineers

Common issues include:

Documentation lagging behind code changes, causing inconsistencies.

Frequent API modifications requiring manual updates.

Inconsistent terminology across the docs.

Duplicated documentation blocks that cannot be referenced like code.

Code snippets extracted from docs that fail to compile.

Additional problems that are hard to automate:

Inconsistent writing style among contributors, suggesting a document‑review process similar to code review.

Unclear or inaccurate language usage, especially in Chinese.

Documentation Engineering Experience

Documentation engineering focuses on specifying, designing, and maintaining technical documentation (e.g., product specifications) with a documentation‑first mindset.

Documentation‑experience design targets the engineers who write docs, aiming to improve their workflow and overall developer experience.

Developer experience (DX) heavily depends on documentation; a key DX metric is TTFHW (Time To First Hello World), which measures how quickly a newcomer can produce a working example.

Key Elements of an Excellent Documentation Engineering Experience

Edit‑Publish Separation : The editing phase and publishing phase are decoupled, allowing different languages/technologies (e.g., Markdown for authoring, rich formats for output).

Process Automation : Rapid, automated publishing keeps documentation in sync with code and supports frequent releases.

Documentation Formalization : Modern DSLs (customized Markdown) enable multiple output formats via tools like Pandoc (PDF, Word, HTML, etc.).

Open Collaboration : Users can improve docs via pull‑requests, mirroring open‑source contribution workflows.

Versioned Management : All changes are tracked (e.g., with Git), allowing rollback and historical inspection of both edit and release versions.

Because documentation needs differ across languages and frameworks, a one‑size‑fits‑all solution rarely exists.

Scenario‑Based Presentation

Different use‑cases demand distinct documentation formats:

Single‑page guides for quick project bootstrapping.

Modular API/SDK docs that enable independent navigation and SEO benefits.

Interactive docs (e.g., REPLs, component libraries) that let users experiment with zero friction.

…and more.

Documentation Modes

Four primary modes are identified:

Basic Mode – Documentation as Code

Documentation is written in a domain‑specific language (DSL) resembling source code and managed with version control, CI pipelines, and deployment tools. Examples include Rustdoc, Julia’s DocumenterTools, and other language‑specific doc generators.

Typical workflow:

Design a documentation DSL for extensibility.

Implement documentation tests to verify correctness.

Build an open‑collaboration platform (e.g., Git‑based PR workflow).

Documentation Tests – Consistency Checks

Documentation tests ensure that code snippets embedded in docs compile and run correctly. In Rustdoc, code blocks are extracted, compiled, and executed; success indicates accurate documentation.

# Examples
```rust
assert_eq!(2 + 2, 4);
```

Generated test harness (simplified):

#![allow(unused)]
fn main() {
    #[allow(non_snake_case)]
    fn _doctest_main__some_unique_name() {
        assert_eq!(2 + 2, 4);
    }
    _doctest_main__some_unique_name();
}

Testing steps:

Parse Markdown, locate Rust code blocks.

Transform blocks into compilable snippets.

Compile; failure indicates a documentation error.

Run the compiled tests.

Executable Documentation

Docs that are themselves executable code, such as R Markdown, Julia’s DocumenterTools, or Exemd, allow dynamic rendering of results, charts, and data directly within the documentation.

```{r fig.show='animate', dev='jpeg', ffmpeg.format='gif'}
for (i in 1:10) plot(runif(100), ylim = c(0, 1))
```

Flexible Code Blocks

DSL‑driven blocks can reference source files dynamically. Example from a custom "Writing" DSL:

// doc-code: file("src/lib.rs").line()[2, 5]
// Read lines 2‑5 from src/lib.rs
// doc-section: file("src/lib.rs").section("section1")
// Pull the code for section1 from src/lib.rs

Tests as Documentation

Behaviour‑driven specifications (e.g., Cucumber) treat test scenarios as living documentation.

# language: zh-CN
@math
功能: 加法
  加法计算器的验证用例

@sanity
场景: 两个数相加
  假如我已经在计算器里输入 6
  而且我已经在计算器里输入 7
  当我按 "相加" 按钮
  那么我应该在屏幕上看到的结果是 13

Designing the Experience

Metrics to evaluate documentation quality and developer experience include:

TTFHW – time to first hello world.

Number of PRs and Issues related to documentation (lower counts indicate maturity).

API coverage – proportion of APIs with corresponding documentation/tests.

SEO visibility – how easily developers discover the docs via search engines.

Web performance – page load speed for online documentation.

Subjective factors such as visual appeal and usability, though harder to quantify.

Improvement steps (adapted from internal DX guidelines):

Define value‑measurement dimensions.

Map them to user journeys and break them into quantifiable indicators.

Establish an experience‑measurement framework.

Perform measurement and diagnosis.

Analyze results and iterate on experience enhancements.

Build a governance mechanism for continuous experience management.

References

API Library Documentation Support: Design for Major Programming Languages

DocumenterTools (Julia)

swift‑doc

Dokka (Kotlin)

librustdoc (Rust)

Digital Technology Strategy: Developer Experience – The "Last Mile" of Internal Tools