How to Build a Simple Perl Compiler: Parsing, AST, and Code Generation

Why Build a Small Language Compiler?

In everyday development we often invent tiny domain‑specific languages to simplify complex problems, and we need a compiler to translate those languages into a form a machine can understand, such as a high‑level language or a serialized data structure.

Compiler Components

A typical compiler consists of three parts: a parser, an abstract syntax tree (AST), and a code emitter.

Parser

For Perl 5, Damian Conway’s Parse::RecDescent module is a convenient tool; it generates a backtracking recursive‑descent parser and comes with extensive CPAN documentation. A fragment of an SQL grammar written with Parse::RecDescent looks like this:

select_query: 'select' distinct(?) select_arg(s /,/)
        from_clause(?)
        where_clause(?)
        group_by_clause(?)
        order_by_clause(?)
        limit_clause(?)
        { SQL::SelectQuery->new({ ... }) }
    | <error>

While powerful, Parse::RecDescent can be slow on large inputs. In such cases the yacc‑style Parse::Yapp module offers much better performance by generating a standard LALR(1) parser, though it is less flexible.

Abstract Syntax Tree (AST)

Simple nested arrays or hashes can represent tree structures in Perl 5, but for easier post‑processing we often use Moose to define DOM‑like node classes, accepting the extra startup cost. An example AST node for a binary relational expression in a SQL‑like mini‑language is:

package SQL::BinaryRelExpr;
    use Moose;
    use lib 'lib';
    use SQL::ArithExpr;

    extends 'SQL::RelExpr';

    has 'terms',
        is => 'rw',
        isa => 'ArrayRef[SQL::ArithExpr]',
        required => 1;

    has 'operator',
        is => 'rw',
        isa => 'Str',
        required => 1;

    ...
    1;

Code Emitter

Often we write Perl code that walks the AST to emit the target language directly. For more structured generation we can use the Perl Template Toolkit (TT2), which is not limited to HTML/XML—it can produce Lua, C, or even Perl code.

“Write‑the‑program‑that‑writes‑programs” is a powerful paradigm.

Future with Perl 6

Perl 6 offers language‑level support for compiler construction; its regex engine can parse complex grammars like Perl 6 itself, making the development of mini‑languages even more promising.