How WebKit Parses HTML: Decoding, Tokenization, and DOM Tree Construction

The article explains the internal workflow of WKWebView when rendering a web page. When the client app creates a WKWebView , three subprocesses are started: the main process, a rendering process, and a network process. The rendering process receives the HTML byte stream from the network process, decodes it into a character stream, and then parses it into a DOM tree.

Decoding is performed by TextResourceDecoder . It first tries to find a charset declaration in the <meta charset> tag; if found, the appropriate codec is used. Otherwise, the whole byte stream is cached and decoded with the default Windows‑1252 codec after the download finishes.

After decoding, the character stream is tokenized by HTMLTokenizer . The tokenizer implements a state machine (e.g., DataState , TagOpenState , DOCTYPEState ) that emits tokens such as StartTag , EndTag , Comment , and Character . A simplified excerpt of the tokenization loop is:

bool HTMLDocumentParser::pumpTokenizerLoop(SynchronousMode mode, bool parsingFragment, PumpSession& session) {
    do {
        auto token = m_tokenizer.nextToken(m_input.current());
        if (!token) return false;
        constructTreeFromHTMLToken(token);
    } while (!isStopped());
    return false;
}

Each token is handed to HTMLTreeBuilder::processToken , which creates the corresponding DOM node and schedules an insertion task. For example, processing a DOCTYPE token looks like:

void HTMLTreeBuilder::processDoctypeToken(AtomHTMLToken&& token) {
    if (m_insertionMode == InsertionMode::Initial) {
        m_tree.insertDoctype(WTFMove(token));
        m_insertionMode = InsertionMode::BeforeHTML;
    }
}

Insertion tasks are stored in a queue and later executed by HTMLConstructionSite::executeQueuedTasks . The actual insertion is performed by ContainerNode::parserAppendChild , which updates sibling pointers and the parent‑child relationships.

The resulting in‑memory DOM tree differs from the logical tree: each element keeps only firstChild and lastChild pointers, while sibling nodes are linked via nextSibling and previousSibling . This representation reduces the need for frequent memory allocations when a node has many children.

Overall, the article provides a detailed, step‑by‑step view of how WebKit decodes HTML, tokenizes it, builds the DOM tree, and manages memory for the resulting structure.