Build a Shared AI Testing Framework That Compounds Team Knowledge
This guide shows how to turn scattered AI contexts and workflows into a shared testing framework using HQ, starting with a weekly intelligence worker that retrieves meeting notes, project status, and decisions, then codifies policies, packages reusable workers, and creates a review-sync loop so every correction improves the team's baseline.
1. Build an Environment Around the Model
Models can reason, write, and call tools, but they need an environment that explains how your company works. A useful framework answers five questions: what the AI knows, how it finds relevant context, which rules it must follow, what repeatable work it can execute, and how each run improves the next. A long system prompt can answer some of these for one session; a company-level framework makes the answers structured, persistent, and available to everyone. Knowledge becomes searchable, rules survive chat endings, tools have boundaries, work leaves artifacts, and accepted workflows become reusable instead of disappearing when a conversation closes. That is why the same model feels completely different at two companies — the model may be identical, but the working environment is not. The model provides intelligence; the framework provides the company.
2. Validate the Execution Framework with One Real Workflow
Trying to model the entire company upfront can waste weeks on context gathering without proving the framework improves any concrete work. Start with a workflow that has four traits: it happens frequently, has clear boundaries, relies on company context, and humans can quickly judge the output. A weekly company intelligence briefing fits well. Inputs already exist but are scattered across meeting notes, project files, decisions, and people's heads. The output is useful to the whole team, and a founder can quickly verify accuracy.
Define the contract before building the executor.
Inputs
Meetings from the past 7 days
Current project status
Decisions, commitments, and open issues
Risks and blocked work
Process
Retrieve relevant sources
Verify every factual claim
Surface contradictions and missing information
Synthesize company-level changes
Output
Decisions made
Progress by project
Risks and blockers
Next-week commitments
Source list
Boundaries
Draft only
Stop before distribution for human review
If the workflow still changes every time a human does it, keep it manual. A process should become repeatable before it becomes shared infrastructure.
3. Give the AI Persistent Company Memory
Start from the current HQ guided setup (https://hqforwork.com/getting-started). Install HQ, create a company workspace, and open HQ as the current working directory in the AI tools your team already uses. The open-source quick start (https://github.com/indigoai-us/hq-core) requires only one command:
npx create-hq
Now add only the context the weekly briefing needs. Begin with this structure:
HQ
companies
your-company
company-brief.md
knowledge: decisions and playbooks
sources: meetings
signals
people
projects
policies: weekly-intelligence.md
workers: weekly-intelligence
From here, add deeper knowledge, skills, automations, and company-specific workers only when a real workflow demands them. The company brief explains what the company does, how it makes money, and what matters most right now. Projects hold current status. Decisions record why the team chose a certain path. People files make role ownership visible. Meeting intelligence captures commitments, risks, issues, and decisions that never make it into formal docs.
Don't stuff the entire company into every prompt. HQ's charter gives agents a map of where knowledge, policies, projects, and workers live. Agents follow the map and retrieve the deeper sources a task needs. Give agents a small, stable entry point and serve deeper context on demand. This keeps company memory available at all times without exhausting the context window before work begins.
4. Turn Company Judgment into Policy
Knowledge tells the executor what happened. Policy tells it how the company expects the work to be handled. Create companies/your-company/policies/weekly-intelligence.md:
Weekly Intelligence Policy
Every factual claim must be backed by a source.
Surface contradictory evidence; never silently resolve it.
Report risks at the urgency level of the original source.
Flag missing, stale, or uncertain information.
Never include confidential or cross-company context.
Must stop before distribution for human approval.
Keep the first policy version short enough that people will actually maintain it. Control has three layers: instructions ask agents to follow a preference; policies make rules persist across sessions and people; hooks or mechanical checks block actions when the cost of failure is high. "Cite your sources" can start as a policy; "never send without approval" should be enforced at the action boundary. Don't try to describe every possible behavior — only encode the few invariants that should persist across every model, teammate, and project.
5. Package the Workflow as a Shared Worker
Now turn the accepted process into a reusable HQ worker. Run /newworker and give it a narrowly scoped task. A generic company analyst agent sounds useful but is hard to test and easy to misuse. The weekly intelligence worker has clear inputs, outputs, and stop conditions.
Worker Spec:
Name: weekly-intelligence
Purpose: Produce a sourced weekly company brief for human review.
Allowed Sources
Company brief
Meetings from the last 7 days
Current projects
Decisions and commitments
Process
Confirm the reporting window.
Retrieve allowed sources.
Extract decisions, progress, risks, and commitments.
Verify claims against source material.
Flag contradictions, gaps, and stale information.
Write the brief in the required format.
Required Output
Executive summary
Decisions made
Project progress
Risks and blockers
Next-week commitments
Open questions
Source list
Must Never
Invent missing facts
Read another company's context
Leak confidential information
Send or publish the brief
Done When: Every claim is sourced or marked uncertain, and the draft is ready for human review.
Company knowledge supplies facts. Policy supplies judgment. The worker supplies a repeatable process. A prompt can generate one useful brief; a worker lets another teammate use the same method next Friday.
6. Make Every Correction Improve the Infrastructure
Don't treat the first successful run as finished infrastructure. Run the worker, review the brief, and diagnose each correction at the right layer:
Missing fact → Improve company knowledge.
Wrong context → Improve routing and resource descriptions.
Repeated mistake → Improve the worker's skill.
Unsafe behavior → Improve policy or hooks.
Weak deliverable → Improve the output contract.
Stale information → Improve knowledge maintenance.
If the worker missed a decision because the meeting was never captured, rewriting the prompt won't fix the system — fix the knowledge path. If it consistently buries risks under minor updates, make the output contract explicit. If someone asks it to distribute the brief without approval, harden the policy and the operational gate. The real question is: Which part of the environment allowed this error? Fix that layer, then rerun the same example. The correction should outlive the output that exposed the problem. This is how human judgment compounds: you teach the system once, then serve the improved behavior to every subsequent run instead of repeating the same correction in private chats.
7. Make the Testing Framework Smarter Every Time the Team Uses It
When the brief, policy, and worker pass review, run /hq-sync. This is where HQ becomes multiplayer for AI. Sales can turn objection handling into a skill. Support can codify escalation rules. Operations can refine a report. Engineering can add a review gate. When each contribution passes review, it syncs to Main and becomes part of the company's shared testing framework.
The next teammate inherits the company-verified context, rules, skills, workers, and automations. They don't need the original chat logs or prompts. They open HQ in their preferred AI tool and continue from the improved version. This is HQ's compounding loop: use the framework, improve one layer, review it, sync it, and raise everyone's starting point.
The model can be Claude, Codex, ChatGPT, or an open-source model. The company layer keeps getting smarter underneath. Company isolation still applies — sync must not erase tenant boundaries, bypass permissions, or put secrets into shared files. A shared execution framework only works when the boundaries around "shared" stay explicit. Shared learning also raises the stakes: a weak instruction now affects everyone, so treat Main like production. Review every contribution before sync. Keep policy scope narrow. Test workers with real examples. As the setup grows, use /harness-audit to check context efficiency, quality gates, persistence, search, and security.
Once a change passes review, sync it. Then pick the next repeatable workflow and improve the shared baseline again. The full progression is: Memory → Context → Policy → Worker → Review → Team Defaults . Start with one recurring job this week. Define its inputs, outputs, and approval boundaries. Run it manually, correct it, then turn the accepted method into a worker the team can share. Models will keep changing. Your company's memory, rules, and best ways of working shouldn't reset with them.
If you want to build a shared execution framework for your team, try HQ (https://hqforwork.com/).
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
