How a 500‑Million‑Parameter ChatGPT Clone Runs Inside Minecraft’s Redstone

CraftGPT Overview

CraftGPT is a proof‑of‑concept language model built entirely inside Minecraft using Redstone logic circuits. The implementation occupies a three‑dimensional region of 1020 × 260 × 1656 blocks, which corresponds to 4.39 × 10⁸ individual Redstone components. The model contains roughly 5 million parameters (embedding dimension = 240, six transformer layers).

Redstone as a Turing‑Complete Logic System

Redstone provides the following basic components, each of which can be combined to realise arbitrary Boolean circuits:

Redstone Dust : carries a signal with strength 0‑15, decreasing by one per block.

Redstone Torch : constant power source and inverter (NOT gate).

Repeater : signal amplification and programmable delay.

Comparator : compares two signal strengths and can perform simple arithmetic.

Piston : moves blocks when powered, enabling memory cells.

Switches (buttons, levers, pressure plates): external inputs.

Using these primitives, the following logic gates are realised:

NOT : a torch on a block extinguishes when the block receives power (input 1 → output 0) and lights when the block is unpowered (input 0 → output 1).

OR : two input lines are merged; any active line drives the output high.

AND : constructed via De Morgan’s law – two NOT gates feeding an OR gate (A AND B = NOT(NOT A OR NOT B)).

With NOT, OR, and AND, any combinational or sequential circuit can be built, and the community has already demonstrated full adders, registers, counters, RAM, 8‑bit CPUs, and even 16‑bit computers inside Minecraft.

CraftGPT Architecture: From Input to Output

Tokenizer : Input text is tokenised into integer IDs (e.g., "Hello world" → [15496, 995]). In CraftGPT the token‑to‑ID mapping is stored in a Redstone ROM that acts as a lookup table.

Embedding Layer : Each token ID indexes a 240‑dimensional vector from a 240 × vocab_size matrix stored in Redstone memory. Positional encodings are added to the vector.

Transformer Stack (6 identical layers) – each layer consists of:

Multi‑Head Attention

Three matrix multiplications (each 240 × 240) produce Query (Q), Key (K), and Value (V) tensors.

Attention scores are computed as Q × Kᵀ.

Softmax normalisation (the most expensive operation in fixed‑point Redstone arithmetic).

The scores weight the V vectors: Attention × V.

A KV cache stores previously computed K and V to avoid recomputation for earlier tokens.

Feed‑Forward Network

First linear projection expands 240‑dimensional vectors to 960 dimensions ( 240 × 960 multiplication).

ReLU activation zeroes negative values.

Second linear projection compresses back to 240 dimensions ( 960 × 240 multiplication).

Layer Normalisation

Mean and variance of the 240‑dimensional vector are computed.

Each element is normalised as (x − mean) / √variance, which requires division and square‑root operations that are costly in Redstone.

Output Layer : After the six transformer layers, the final 240‑dimensional vector is multiplied by an un‑embedding matrix to produce logits over the vocabulary. The token with the highest probability is emitted as the model’s output.

Computation Cost

Per layer matrix multiplications: four 240 × 240, one 240 × 960, and one 960 × 240 – repeated six times.

Each multiplier is implemented with tens of thousands of basic adders and multipliers, each built from dozens of Redstone logic gates.

The total number of elementary logical operations reaches the order of several hundred million, all represented by the on/off state of Redstone torches.

Running CraftGPT

Download MCHPRS (Minecraft High Performance Redstone Server).

Obtain the CraftGPT world file from its GitHub repository (the repository contains the full Redstone circuit and ROM data).

Prepare a machine with at least 32 GB RAM (64 GB recommended for smoother operation).

Start the MCHPRS server and connect with a Minecraft 1.20.4 client.

Enter a prompt in the in‑game chat. The model processes the input using the Redstone circuit and returns a response after several hours of computation.

Limitations observed by the author include a context window of only 64 tokens, frequent off‑topic or grammatically incorrect outputs, and occasional garbled text. Nevertheless, CraftGPT demonstrates that a full transformer‑style neural network can be realised within a sandbox game’s native logic system.