Skip to content
Build a Shared AI Testing Framework That Compounds Team Knowledge

Build a Shared AI Testing Framework That Compounds Team Knowledge

This guide shows how to turn scattered AI contexts and workflows into a shared testing framework using HQ, starting with a weekly intelligence worker that retrieves meeting notes, project status, and decisions, then codifies policies, packages reusable workers, and creates a review-sync loop so every correction improves the team's baseline.

ThinkInAI Community
ThinkInAI Community
ThinkInAI Community

1. Build an Environment Around the Model

Models can reason, write, and call tools, but they need an environment that explains how your company works. A useful framework answers five questions: what the AI knows, how it finds relevant context, which rules it must follow, what repeatable work it can execute, and how each run improves the next. A long system prompt can answer some of these for one session; a company-level framework makes the answers structured, persistent, and available to everyone. Knowledge becomes searchable, rules survive chat endings, tools have boundaries, work leaves artifacts, and accepted workflows become reusable instead of disappearing when a conversation closes. That is why the same model feels completely different at two companies — the model may be identical, but the working environment is not. The model provides intelligence; the framework provides the company.

2. Validate the Execution Framework with One Real Workflow

Trying to model the entire company upfront can waste weeks on context gathering without proving the framework improves any concrete work. Start with a workflow that has four traits: it happens frequently, has clear boundaries, relies on company context, and humans can quickly judge the output. A weekly company intelligence briefing fits well. Inputs already exist but are scattered across meeting notes, project files, decisions, and people's heads. The output is useful to the whole team, and a founder can quickly verify accuracy.

Define the contract before building the executor.

Inputs

Meetings from the past 7 days

Current project status

Decisions, commitments, and open issues

Risks and blocked work

Process

Retrieve relevant sources

Verify every factual claim

Surface contradictions and missing information

Synthesize company-level changes

Output

Decisions made

Progress by project

Risks and blockers

Next-week commitments

Source list

Boundaries

Draft only

Stop before distribution for human review

If the workflow still changes every time a human does it, keep it manual. A process should become repeatable before it becomes shared infrastructure.

3. Give the AI Persistent Company Memory

Start from the current HQ guided setup (https://hqforwork.com/getting-started). Install HQ, create a company workspace, and open HQ as the current working directory in the AI tools your team already uses. The open-source quick start (https://github.com/indigoai-us/hq-core) requires only one command:

npx create-hq

Now add only the context the weekly briefing needs. Begin with this structure:

HQ

companies

your-company

company-brief.md

knowledge: decisions and playbooks

sources: meetings

signals

people

projects

policies: weekly-intelligence.md

workers: weekly-intelligence

From here, add deeper knowledge, skills, automations, and company-specific workers only when a real workflow demands them. The company brief explains what the company does, how it makes money, and what matters most right now. Projects hold current status. Decisions record why the team chose a certain path. People files make role ownership visible. Meeting intelligence captures commitments, risks, issues, and decisions that never make it into formal docs.

Don't stuff the entire company into every prompt. HQ's charter gives agents a map of where knowledge, policies, projects, and workers live. Agents follow the map and retrieve the deeper sources a task needs. Give agents a small, stable entry point and serve deeper context on demand. This keeps company memory available at all times without exhausting the context window before work begins.

4. Turn Company Judgment into Policy

Knowledge tells the executor what happened. Policy tells it how the company expects the work to be handled. Create companies/your-company/policies/weekly-intelligence.md:

Weekly Intelligence Policy

Every factual claim must be backed by a source.

Surface contradictory evidence; never silently resolve it.

Report risks at the urgency level of the original source.

Flag missing, stale, or uncertain information.

Never include confidential or cross-company context.

Must stop before distribution for human approval.

Keep the first policy version short enough that people will actually maintain it. Control has three layers: instructions ask agents to follow a preference; policies make rules persist across sessions and people; hooks or mechanical checks block actions when the cost of failure is high. "Cite your sources" can start as a policy; "never send without approval" should be enforced at the action boundary. Don't try to describe every possible behavior — only encode the few invariants that should persist across every model, teammate, and project.

5. Package the Workflow as a Shared Worker

Now turn the accepted process into a reusable HQ worker. Run /newworker and give it a narrowly scoped task. A generic company analyst agent sounds useful but is hard to test and easy to misuse. The weekly intelligence worker has clear inputs, outputs, and stop conditions.

Worker Spec:

Name: weekly-intelligence

Purpose: Produce a sourced weekly company brief for human review.

Allowed Sources

Company brief

Meetings from the last 7 days

Current projects

Decisions and commitments

Process

Confirm the reporting window.

Retrieve allowed sources.

Extract decisions, progress, risks, and commitments.

Verify claims against source material.

Flag contradictions, gaps, and stale information.

Write the brief in the required format.

Required Output

Executive summary

Decisions made

Project progress

Risks and blockers

Next-week commitments

Open questions

Source list

Must Never

Invent missing facts

Read another company's context

Leak confidential information

Send or publish the brief

Done When: Every claim is sourced or marked uncertain, and the draft is ready for human review.

Company knowledge supplies facts. Policy supplies judgment. The worker supplies a repeatable process. A prompt can generate one useful brief; a worker lets another teammate use the same method next Friday.

6. Make Every Correction Improve the Infrastructure

Don't treat the first successful run as finished infrastructure. Run the worker, review the brief, and diagnose each correction at the right layer:

Missing fact → Improve company knowledge.

Wrong context → Improve routing and resource descriptions.

Repeated mistake → Improve the worker's skill.

Unsafe behavior → Improve policy or hooks.

Weak deliverable → Improve the output contract.

Stale information → Improve knowledge maintenance.

If the worker missed a decision because the meeting was never captured, rewriting the prompt won't fix the system — fix the knowledge path. If it consistently buries risks under minor updates, make the output contract explicit. If someone asks it to distribute the brief without approval, harden the policy and the operational gate. The real question is: Which part of the environment allowed this error? Fix that layer, then rerun the same example. The correction should outlive the output that exposed the problem. This is how human judgment compounds: you teach the system once, then serve the improved behavior to every subsequent run instead of repeating the same correction in private chats.

7. Make the Testing Framework Smarter Every Time the Team Uses It

When the brief, policy, and worker pass review, run /hq-sync. This is where HQ becomes multiplayer for AI. Sales can turn objection handling into a skill. Support can codify escalation rules. Operations can refine a report. Engineering can add a review gate. When each contribution passes review, it syncs to Main and becomes part of the company's shared testing framework.

The next teammate inherits the company-verified context, rules, skills, workers, and automations. They don't need the original chat logs or prompts. They open HQ in their preferred AI tool and continue from the improved version. This is HQ's compounding loop: use the framework, improve one layer, review it, sync it, and raise everyone's starting point.

Article diagram 1
Article diagram 1
Article diagram 2
Article diagram 2
Article diagram 3
Article diagram 3
Article diagram 4
Article diagram 4
Article diagram 5
Article diagram 5

The model can be Claude, Codex, ChatGPT, or an open-source model. The company layer keeps getting smarter underneath. Company isolation still applies — sync must not erase tenant boundaries, bypass permissions, or put secrets into shared files. A shared execution framework only works when the boundaries around "shared" stay explicit. Shared learning also raises the stakes: a weak instruction now affects everyone, so treat Main like production. Review every contribution before sync. Keep policy scope narrow. Test workers with real examples. As the setup grows, use /harness-audit to check context efficiency, quality gates, persistence, search, and security.

Once a change passes review, sync it. Then pick the next repeatable workflow and improve the shared baseline again. The full progression is: Memory → Context → Policy → Worker → Review → Team Defaults . Start with one recurring job this week. Define its inputs, outputs, and approval boundaries. Run it manually, correct it, then turn the accepted method into a worker the team can share. Models will keep changing. Your company's memory, rules, and best ways of working shouldn't reset with them.

If you want to build a shared execution framework for your team, try HQ (https://hqforwork.com/).

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

AI agentsteam collaborationpolicy as codeAI testing frameworkcompound learningHQshared AI infrastructureweekly intelligence
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.