instinct: A Confidence‑Based Self‑Learning Memory System for AI Agents

Problem

AI coding agents such as Claude Code, Cursor, and GitHub Copilot perform well within a single session but start from scratch in each new session, lacking continuity and the ability to recognise recurring situations.

Existing work‑arounds rely on human‑maintained files (e.g., CLAUDE.md, system prompts) that make the human the memory carrier while the agent merely executes commands.

Core Idea: Confidence‑Based Learning

instinct adopts a habit‑formation‑inspired loop: observe → repeat → mature → suggest. Each observed pattern starts with confidence 1; repeated observations increment confidence. When confidence crosses the first threshold the pattern becomes mature and appears in the suggestion list; crossing a higher threshold promotes it to a rule that can be auto‑applied.

Confidence 1‑4:   raw      (observed, not actionable)
Confidence 5‑9:   mature   (ready to suggest)
Confidence 10+:  rule     (strong enough to auto‑apply)

A pattern is considered “ready” solely based on its confidence score—no human judgement is required.

How It Works

1. Observe

When a recurring pattern (e.g., a sequence of tool calls, a coding‑style preference, or a frequent fix) is detected, instinct records it as a raw observation with confidence 1.

instinct observe "seq:lint->fix->test"
instinct observe "pref:style=black" --cat preference
instinct observe "fix:missing-import" --cat fix_pattern

Pattern categories are distinguished by prefixes:

seq:   → Action sequence (e.g. seq:lint->fix->lint)
pref:  → User preference (e.g. pref:stdlib-first)
fix:   → Recurring fix (e.g. fix:missing-import)
combo: → Tools used together (e.g. combo:pytest+coverage)

Each observation is an upsert: if the pattern exists, confidence is incremented; otherwise a new row with confidence 1 is created.

2. Consolidate (Merge)

At the end of a session—or periodically—the consolidate command promotes patterns whose confidence has reached the thresholds.

instinct consolidate
# Promoted to mature: 3
# Promoted to rule: 1
# Total instincts: 12

Learning happens in this step; no manual intervention is needed.

3. Suggest

When a new session starts, the agent requests suggestions. Only mature and rule patterns are returned; raw observations remain silent until their confidence grows.

instinct suggest
#   seq:test->fix->test    conf=8  [mature]  sequence
#   pref:stdlib-first      conf=12 [rule]    preference
# 2 suggestions

The suggestion list can contain both global patterns and project‑specific patterns.

Architecture

Why SQLite?

Alternative storage options (JSON files, Redis, simple key‑value stores) were evaluated. SQLite was chosen because:

SQL queries can filter by confidence, category, or project.

ACID transactions guarantee state consistency.

No daemon or extra configuration is required; the store is a single file.

Learning history can be migrated simply by copying the file.

The core upsert query merges creation and update in one statement:

INSERT INTO instincts (pattern, category, confidence, ...)
VALUES (?, ?, 1, ...)
ON CONFLICT(pattern) DO UPDATE SET
    confidence = confidence + 1,
    last_seen = excluded.last_seen

Why MCP?

Model Context Protocol (MCP) is an open standard that connects AI agents to external tools. By implementing instinct as an MCP server, it works out‑of‑the‑box with any MCP‑compatible agent such as Claude Code, Cursor, or Goose.

{
  "mcpServers": {
    "instinct": {
      "command": "instinct",
      "args": ["serve"]
    }
  }
}

Adding these lines to an agent’s configuration enables seamless integration.

Project‑Aware Learning

Not all patterns are universally applicable. instinct scopes patterns using a project fingerprint derived from the project’s directory path:

def project_fingerprint(path=None):
    p = Path(path or Path.cwd()).resolve()
    return hashlib.sha256(str(p).encode()).hexdigest()[:12]

Patterns with a non‑empty project field apply only to that project; patterns with an empty project field are global.

Decay: Forgetting Stale Patterns

Patterns that have not been observed for a configurable period lose confidence and are eventually removed, keeping the memory timely. instinct decay --days 90 Any pattern unseen for more than 90 days has its confidence decreased by 1; when confidence reaches 0 the pattern is deleted.

Real‑World Usage

Use observe to record patterns you notice. Use suggest to get mature patterns that should guide your behavior. Run consolidate periodically to auto‑promote high‑confidence patterns.

After a few sessions, the agent builds its own operational manual:

seq:test->fix->test          conf=8   [mature] — Always re‑run tests after fixes
pref:stdlib-first           conf=12  [rule]   — Prefer stdlib over third‑party
fix:missing-init            conf=6   [mature] — Check for __init__.py in new packages
combo:pytest+coverage       conf=5   [mature] — Always run coverage with tests

Database Schema (SQLite)

CREATE TABLE instincts (
    pattern    TEXT PRIMARY KEY,
    category   TEXT NOT NULL DEFAULT 'sequence',
    confidence INTEGER NOT NULL DEFAULT 1,
    first_seen TEXT NOT NULL,
    last_seen  TEXT NOT NULL,
    source     TEXT NOT NULL DEFAULT '',
    project    TEXT NOT NULL DEFAULT '',
    promoted   INTEGER NOT NULL DEFAULT 0,
    metadata   TEXT NOT NULL DEFAULT '{}'
)

The promoted column encodes maturity (0 = raw, 1 = mature, 2 = rule). The metadata column stores arbitrary JSON for future extensions.

Python API

from instinct.store import InstinctStore

store = InstinctStore()  # default: ~/.instinct/instinct.db

# Observe patterns
store.observe("seq:test->fix->test", source="claude-code")
store.observe("seq:test->fix->test")  # confidence now = 2

# Query
suggestions = store.suggest()               # get mature patterns
store.list(min_confidence=3)               # filter by confidence
store.search("test")                      # full‑text search

# Lifecycle
store.consolidate()                        # auto‑promote
store.decay(days_inactive=90)             # forget stale patterns
stats = store.stats()                       # summary statistics

# Export
rules = store.export_rules()               # only confidence >= 10
store.close()

The API allows instinct to be embedded in custom toolchains, CI pipelines, or analytics dashboards.

Local Testing

pip install instinct-mcp

# Record your first pattern
instinct observe "seq:your->first->pattern"

# See what the system has learned
instinct suggest

# Start the MCP server for agent integration
instinct serve

The code is fully open‑source and published on PyPI.

Repository: https://github.com/yakuphanycl/instinct