Explore tinygrad: A Minimalist Deep Learning Framework Under 1000 Lines

tinygrad is an open‑source minimalist autograd tensor library created by George Hotz, positioned between PyTorch and micrograd, with fewer than 1000 lines of code and over 1400 GitHub stars.

It can be installed via pip3 install tinygrad --upgrade and provides a PyTorch‑like API for building and training neural networks.

Basic usage example:

from tinygrad.tensor import Tensor
x = Tensor.eye(3)
y = Tensor([[2.0,0,-2.0]])
z = y.matmul(x).sum()
z.backward()
print(x.grad)  # dz/dx
print(y.grad)  # dz/dy

Optimizers such as SGD, RMSprop, and Adam are available in tinygrad.optim, allowing full training loops similar to PyTorch.

GPU acceleration is supported through PyOpenCL, e.g.:

from tinygrad.tensor import Tensor
(Tensor.ones(4,4).cuda() + Tensor.ones(4,4).cuda()).cpu()

tinygrad can run full EfficientNet inference on ImageNet images:

ipython3 examples/efficientnet.py https://upload.wikimedia.org/wikipedia/commons/4/41/Chicken.jpg

Tests can be executed with python -m pytest. Future plans include adding language and detection models to further reduce code size and improve speed.