Explore 7 Lightweight Python Databases for Small Projects and Learning

Python, as a high‑level language with strong community support, is used to implement many types of databases. This article highlights several uncommon Python databases that can deepen your understanding of database concepts by studying their source code.

PickleDB

PickleDB is a lightweight key‑value store written in Python that uses JSON files for persistence, offering a simple API suitable for small projects or quick persistence needs.

Key Features

Lightweight and not suited for large or high‑concurrency workloads.

Simple dictionary‑like API.

JSON‑based storage for easy reading and modification.

Data persisted to disk.

No external dependencies; runs with just Python.

Use Cases

Configuration storage for applications.

Small projects requiring quick setup.

Learning and teaching basic database operations.

Temporary data storage during development or testing.

Cautions

Performance and scalability are limited; not ideal for large datasets or high concurrency.

JSON files lack encryption and access control, making them unsuitable for sensitive data.

Basic Usage

import pickledb

# Create or open a database
db = pickledb.load('example.db', auto_dump=True)

# Insert data
db.set('key1', 'value1')

# Retrieve data
value = db.get('key1')
print(value)  # Output: value1

# Check if key exists
exists = db.exists('key1')
print(exists)  # Output: True

# Delete data
db.rem('key1')

# Get all keys
keys = db.getall()
print(keys)

# Force dump to file
db.dump()

TinyDB

TinyDB is a lightweight, document‑oriented NoSQL database written in Python, storing data in JSON files and offering a simple API with optional plugins for extensibility.

Key Features

Pure Python, no external dependencies.

Document‑oriented storage with flexible nested structures.

Simple, Python‑like API for CRUD operations.

Powerful query language supporting complex conditions and regex.

Basic transaction support.

Use Cases

Small projects needing quick, embedded storage.

Embedded applications and desktop tools.

Prototyping and early‑stage development.

Configuration storage requiring persistence.

Cautions

Performance and scalability are limited for large datasets.

File‑based storage can degrade as file size grows.

Basic Usage

from tinydb import TinyDB, Query

# Create or open a database
db = TinyDB('db.json')

# Insert data
db.insert({'name': 'John', 'age': 22})
db.insert({'name': 'Jane', 'age': 25})

# Query data
User = Query()
result = db.search(User.name == 'John')
print(result)  # Output: [{'name': 'John', 'age': 22}]

# Update data
db.update({'age': 23}, User.name == 'John')

# Delete data
db.remove(User.name == 'Jane')

# Retrieve all data
all_data = db.all()
print(all_data)

# Close the database
db.close()

ZODB

ZODB (Zope Object Database) is an object‑oriented database for persisting Python objects directly, eliminating the need for relational mapping and supporting ACID transactions.

Key Features

Object‑oriented storage of Python objects.

Transparent persistence handling serialization automatically.

ACID transaction support.

Versioning and history tracking.

Extensible storage backends (file, memory, network).

Schema‑less design.

Use Cases

Persisting complex object graphs in applications.

Python‑centric projects needing tight integration.

Applications requiring reliable transaction guarantees.

Cautions

Performance and scalability are suited for medium‑sized workloads.

Learning curve for developers accustomed to relational databases.

Smaller community and ecosystem.

Basic Usage

import transaction
from ZODB import FileStorage, DB

# Define a persistent class
class Person(persistent.Persistent):
    def __init__(self, name, age):
        self.name = name
        self.age = age

# Set up storage and database
storage = FileStorage.FileStorage('mydata.fs')
db = DB(storage)
connection = db.open()
root = connection.root()

# Store an object
root['person'] = Person('John Doe', 30)
transaction.commit()

# Retrieve the object
person = root['person']
print(person.name, person.age)

# Clean up
connection.close()
db.close()
storage.close()

Durus

Durus is a lightweight, object‑oriented persistence system written in Python, offering simple transaction support and a minimal API for small projects.

Key Features

Object‑oriented storage of Python objects.

Persistent storage to disk.

Basic transaction support.

Simple, easy‑to‑learn API.

Very lightweight with no complex configuration.

Use Cases

Small projects needing simple persistence.

Python applications that store objects directly.

Rapid prototyping where a full‑featured DB is unnecessary.

Cautions

Limited performance and scalability for large datasets.

Feature set is minimal; no advanced querying or indexing.

Small community support.

Basic Usage

from durus.persistent import Persistent
from durus.connection import Connection
from durus.storage import FileStorage

# Define a persistent class
class Person(Persistent):
    def __init__(self, name, age):
        self.name = name
        self.age = age

# Create storage and connection
storage = FileStorage('mydata.durus')
connection = Connection(storage)
root = connection.get_root()

# Store an object
root['person'] = Person('John Doe', 30)
connection.commit()

# Retrieve the object
person = root['person']
print(person.name, person.age)

# Clean up
connection.close()
storage.close()

Buzhug

Buzhug is a pure‑Python lightweight database offering a simple, SQL‑like query language, ideal for small projects and teaching basic database concepts.

Key Features

Implemented entirely in Python, no external dependencies.

SQL‑like query syntax.

Lightweight and easy to use.

Schema‑less, allowing flexible fields.

Use Cases

Small projects requiring simple data storage.

Educational settings for learning SQL basics.

Rapid prototyping before moving to a full‑featured DB.

Cautions

Performance and functionality are limited to small datasets.

Small community and ecosystem.

Lacks advanced transaction and indexing features.

Basic Usage

from buzhug import Base

# Create or open a database
db = Base('people').create(('name', str), ('age', int))

# Insert data
db.insert(name='John Doe', age=30)
db.insert(name='Jane Doe', age=25)

# Query data
for person in db.select():
    print(person.name, person.age)

# Update data
db.update(db.name == 'John Doe', age=31)

# Delete data
db.delete(db.name == 'Jane Doe')

# Close the database
db.close()

Gadfly

Gadfly is a pure‑Python lightweight relational database that supports standard SQL, making it suitable for teaching and small applications without requiring an external server.

Key Features

Pure Python implementation, runs anywhere Python is available.

Supports standard SQL queries.

Lightweight and embeddable; no server required.

Use Cases

Teaching SQL and basic database concepts.

Small projects needing simple relational storage.

Rapid prototyping before migrating to a more robust DB.

Cautions

Performance and feature set are limited for large or high‑throughput workloads.

Small community and limited ongoing development.

Potential compatibility adjustments needed for modern Python versions.

Basic Usage

import gadfly

# Create or connect to a database
connection = gadfly('mydb', 'mydb_directory')
cursor = connection.cursor()

# Create a table
cursor.execute("CREATE TABLE people (name VARCHAR, age INTEGER)")

# Insert data
cursor.execute("INSERT INTO people (name, age) VALUES ('John Doe', 30)")
cursor.execute("INSERT INTO people (name, age) VALUES ('Jane Doe', 25)")

# Query data
cursor.execute("SELECT * FROM people")
for row in cursor.fetchall():
    print(row)

# Update data
cursor.execute("UPDATE people SET age = 31 WHERE name = 'John Doe'")

# Delete data
cursor.execute("DELETE FROM people WHERE name = 'Jane Doe'")

# Commit and close
connection.commit()
connection.close()

PyTables

PyTables is an open‑source library for managing large scientific datasets using the HDF5 file format, offering efficient storage, compression, hierarchical organization, and seamless NumPy integration.

Key Features

Built on HDF5, a mature format for large data.

Supports multiple compression algorithms (zlib, LZO, BZIP2).

Hierarchical data organization (groups and tables).

Handles datasets that exceed memory capacity.

Supports various data types and integrates with NumPy.

Powerful query capabilities.

Installation

Install via pip:

pip install tables

Basic Usage

import numpy as np
import tables

# Create an HDF5 file
with tables.open_file('example.h5', mode='w') as file:
    # Create a group
    group = file.create_group('/', 'data_group', 'Data Group')

    # Define a structured NumPy array
    data = np.array([(1, b'Hello'), (2, b'World')],
                    dtype=[('number', 'i4'), ('word', 'S10')])

    # Create a table
    table = file.create_table(group, 'example_table', description=data.dtype,
                              title='Example Table')

    # Insert rows
    row = table.row
    for item in data:
        row['number'] = item['number']
        row['word'] = item['word']
        row.append()
    table.flush()

    # Query rows where number > 1
    for row in table.where('number > 1'):
        print(row['number'], row['word'].decode('utf-8'))

    # Read all data into a NumPy array
    np_data = table.read()
    print(np_data)

Use Cases

Scientific computing and data analysis (e.g., climate, genomics, physics simulations).

Managing datasets larger than available RAM.

Data archiving and sharing using the widely supported HDF5 format.

Cautions

Performance tuning may be required based on data layout and compression.

Depends on the external HDF5 library; proper installation is necessary.

Version compatibility of HDF5 files should be considered when moving between environments.