How to Build a Simple Python Template Engine from Scratch

Introduction

Most programs contain a lot of logic and a small amount of textual data, which fits typical programming languages. Some tasks involve minimal logic but massive text data, such as generating HTML for web applications. For these tasks a template engine is a more suitable tool, and this chapter builds a simple one.

The most common text‑heavy task in web apps is generating HTML for browsers. Pages are rarely completely static; they contain dynamic data like usernames, product lists, or news updates. Each page also includes a large amount of static text written by front‑end designers, which they want to keep in familiar HTML form.

To illustrate, suppose we need to generate an HTML page where the user's name, product names, prices, and quantities are dynamic. One naive approach is to concatenate string constants in code, but this quickly becomes messy and hard to maintain, especially as page complexity grows.

Template

A better method is to write the HTML page as a template: mostly static HTML with special markers for dynamic fragments. The demo template looks like this:

The focus is on HTML text with embedded logic, contrasting a document‑centric approach with the earlier logic‑centric Python code.

Using a template engine involves two main functions: parsing the template and rendering it. Rendering replaces dynamic fragments with real data.

Supported Syntax

The engine’s syntax is based on Django. The following constructs are supported:

Insert context data with double curly braces:

Access object attributes or dictionary values using dot notation:

Filters modify values via the pipe character:

Conditional statements:

Loops to include collections:

Comments:

Implementation Overview

The engine performs two main tasks: parsing the template and rendering it.

Manage dynamic context and data sources.

Execute logical elements.

Implement dot access and filter execution.

Parsing can be either an interpretive model (building a data structure representing the template) or a compiled model (generating executable Python code). This engine uses the compiled model: the template is compiled into Python code, which is then executed to produce the final HTML.

Compiling Code

During compilation the template is turned into a Python function called render_function that receives a context dictionary and a do_dots helper for attribute access. The compilation process uses a CodeBuilder class to assemble code lines, manage indentation, and create sections for later insertion.

Key steps include:

Caching frequently used functions like list.append and str for performance.

Creating a buffered output list and flushing it to the result via append_result or extend_result.

Using a stack ( ops_stack) to match control structures ( if, for, etc.).

Splitting the template into tokens with a regular expression and processing each token according to its type.

During token processing, literals are wrapped with repr to preserve quoting, empty tokens are skipped, and after processing the stack is checked to ensure all control structures are closed.

Variables used in the template are collected into all_vars and loop variables into loop_vars. After parsing, code is added to extract these variables from the context into local variables.

Expression Compilation

Expressions are compiled recursively. If an expression contains a pipe ( |), it is split and each part is treated as a function, with the output of one feeding into the next. If it contains dots ( .) but no pipe, the first part is evaluated and subsequent parts are passed to do_dots. Simple variable names are returned directly with a c_ prefix.

Examples: x.y.z becomes do_dots(x, 'y', 'z').

Filters are applied via the pipe syntax.

Rendering

After compilation, rendering is straightforward: merge the initial data with any additional arguments to form the final context, then call the compiled _render_function to obtain the rendered HTML.

The _do_dots helper resolves attribute or key access and calls the value if it is callable, proceeding recursively for chained accesses.

Future Work (TODO)

Template inheritance and inclusion.

Custom tags.

Automatic escaping.

Filter arguments.

Complex control logic such as else and elif.

Loops with multiple loop variables.

Whitespace control.