How I Built Pinecone: From Zero to a Working Compiler

I have been working for six months on a new programming language called Pinecone . It is not yet mature, but it already supports many typical language features such as variables, functions, and user‑defined structs.

When I started, I had no clear direction and was not an expert. I began by reading online resources without strictly following best‑practice advice, yet I managed to produce a complete, working language.

In this article I explain how the source becomes "magic", discuss the trade‑offs I made, and why I made those decisions.

Getting Started

Many developers ask where to begin when creating a new language. I share the steps and decisions that helped me start.

Compiled vs. Interpreted Languages

Languages are generally either compiled or interpreted. Compiled languages translate source code to machine code for fast execution; interpreted languages execute source line‑by‑line, offering flexibility. Pinecone is primarily a compiled language, but it also includes a fully functional interpreter.

Choosing an Implementation Language

I wrote Pinecone in C++ because of its performance and rich feature set. For an interpreted language, implementing it in a compiled language (C, C++, Swift) makes sense to avoid the performance penalty of the interpreter itself. Slower languages like Python or Java are acceptable for interpreted targets, though compile‑time may suffer.

High‑Level Design

A language compiler is usually a pipeline of stages. The first stage receives the entire source file as a string; the final stage produces executable code. Pinecone follows this linear pipeline.

Lexing (Tokenization)

Lexing breaks the source text into tokens such as identifiers, operators, and numbers. I initially considered using Flex, but I kept my own hand‑written lexer because it was only a few hundred lines, bug‑free, and gave me flexibility without adding a dependency.

Parsing

The parser converts the token list into an Abstract Syntax Tree (AST). I wrote a custom parser (about 750 lines) instead of using Bison because it reduces context switches, keeps the build simple, and lets me experiment freely.

Action Tree vs. AST

An Action Tree is similar to LLVM IR and adds context (such as return‑type information) to the AST. This extra information makes execution easier because each node has an execute function that returns a result.

Compilation Choices

I explored three approaches:

Write a custom compiler – ambitious but complex across platforms.

Use LLVM – powerful but heavyweight; I postponed it until I gain more experience.

Transpile to C++ – I built a Pinecone‑to‑C++ transpiler that invokes GCC to produce binaries. It works for most programs, though it is not the most portable solution.

Future Plans

Eventually I intend to add LLVM‑based compilation, but the interpreter remains useful for small scripts, and the C++ transpiler provides a practical performance boost.

Conclusion

I hope Pinecone’s design is clear and helpful. If you want to build your own language, start with an interpreter, choose your lexer and parser freely, and experiment with the pipeline. Building a language sharpens your mental model of programming concepts and makes learning additional languages easier.

For more details see the landing page and the GitHub repository linked below.

Landing page: https://pinecone-lang.herokuapp.com/index.html

GitHub: https://github.com/william01110111/Pinecone