LoongSuite Pilot + Qoder: One Config, Full-Team AI Coding Auditability
The article explains how LoongSuite Pilot normalizes telemetry from 20+ AI coding agents into a unified GenAI event schema with three-level session structure, token breakdowns, cost fields, and provider normalization, while its Qoder plugin achieves near-100% collection coverage via silent installation, self-healing, auto-updates, and parameter-fingerprint-based idempotent upgrades — all driven by a single admin configuration pushed through Qoder's enterprise marketplace.
Background: The Observability Gap in Organizational AI Coding
AI coding has moved beyond individual experimentation: budgets are approved annually, seats purchased per head, token costs settled monthly, and adoption rates plus AI-generated code ratios have become organizational transformation metrics. Since this is a deliberate organizational investment, leaders must answer what the investment returns — but three questions stump most teams: how many AI coding agents are actually in use, how many tokens are consumed daily and at what cost, and which tools truly drive output versus which are merely installed but unused.
Tools like Claude Code, Cursor, Codex, Qwen Code, and Qoder grow organically within teams, each developer choosing differently. The answers lie not in intuition but in data, and that data's usability depends on two independent prerequisites:
Cross-agent data normalization — determines metric comparability. Each agent has different log formats, hook mechanisms, and session structures. Dumping raw logs from a dozen tools yields a dozen incompatible formats, not an aggregatable report.
Collection coverage — whether the collector can be deployed to every dev machine and keep running. This is often treated as an adoption problem but is actually a distribution control problem: how to deliver install packages precisely across the organization, how to converge configuration changes idempotently to every machine, who restarts a crashed collector process, and how to handle gradual rollout and rollback.
Both are necessary but differ in nature. Normalization is a data modeling problem solved inside the collector; once done, all agents and teams benefit. Coverage is a distribution and configuration management problem solved in the delivery channel; it never converges once — next week someone reinstalls their OS, next month a new hire joins, next quarter the reporting config changes. Without a delivery channel, coverage immediately decays.
Most teams get stuck on coverage. Any solution relying on "ask everyone to run an install command" hits four bottlenecks:
Notification unseen : Announcement sent but some miss it, delay, or forget
Install fails : Wrong Node version, nvm conflicts, environment clashes cause uninstall
Installed but not running : "Installed" ≠ "running"; machine reboot or process kill silently stops collection
Config change = redo : Masking policy or reporting params change triggers another full-team notification round
These bottlenecks compound, converging coverage to the early-adopter subset — no matter how perfect normalization is, it only computes a biased sample more precisely.
Collection Foundation — LoongSuite Pilot's Multi-Agent Event Normalization
Auto-Discovery & Collection Deployment — Adapting to Each Agent's Mechanism
LoongSuite Pilot is a telemetry collector running on the developer's machine. It does four things: discovers installed agents, deploys required hooks or plugins, normalizes disparate agent activity into a unified GenAI event schema, and reports to an observability backend.
Pilot detects supported agents via local paths and commands — no user declaration needed — then completes integration per each agent's mechanism. Integration methods vary widely; Pilot internalizes all complexity:
Hook : Claude Code, Codex, Cursor, Qwen Code CLI, Qoder series
Plugin Injection : OpenCode, OpenClaw, MiMo Code
Native Plugin / Extension : Hermes Agent, Pi Coding Agent
YAML patch + local JSONL polling : DeepSeek Harness
Hook wake + transcript listener fallback : WorkBuddy
CLI API polling : Wukong
Current support matrix covers 20 mainstream agents including Qoder full series (IDE / CN / JetBrains / CLI / Work), Cursor and Cursor CLI, Kiro CLI, etc. Agent definitions are declarative configs in the agents.d/ directory; adding a new agent requires no collector core code changes.
Unified Event Schema — Aligning with GenAI Semantic Conventions
This is normalization's core answer and Pilot's central value. Pilot normalizes each agent's native events into a GenAI event field set aligned with GenAI audit-event semantic conventions. Core event types: llm.request: One LLM request: user input, context delta, requested model llm.response: One LLM response: text, reasoning, tool-call intent, finish reason, token usage and cost tool.call / tool.result: One actual tool execution and its return skill.use: Skill, extension, or agent capability invocation tool.approve: User approves tool execution
Key is field completeness. Normalization yields not just "who called which model when" but structured data directly supporting cost and efficiency analysis:
Three-level session structure : gen_ai.session.id (session), gen_ai.turn.id (one user request to final response), gen_ai.step.id (one ReAct loop) — making "how many turns per request, how many steps per turn" computable.
Token breakdown : beyond input/output/total, separately records cache_read and cache_creation input tokens. Cache hit rate directly determines actual spend; merged stats severely overestimate cost.
Cost fields : when pricing info exists, directly provides input, output, cache read/write, and total cost (USD) — downstream no longer maintains a price table.
Performance fields : gen_ai.response.time_to_first_token records first-token latency (nanoseconds), the most critical metric for developer-perceived speed.
Provider normalization : gen_ai.provider.name standardized to anthropic, openai, qwen, deepseek, gcp.vertex_ai, aws.bedrock, etc. Same model via different gateways aggregates together.
Content & deduplication : full input/output messages are opt-in; provides messages_delta (delta vs previous request) and messages_hash (full context hash) enabling deduplication and context-bloat analysis without storing full text.
A normalized event looks like (excerpt, illustrative values):
{
"event.name": "llm.response",
"user.id": "12345",
"gen_ai.agent.type": "claude-code",
"gen_ai.provider.name": "anthropic",
"gen_ai.session.id": "sess-a1b2",
"gen_ai.turn.id": "turn-7",
"gen_ai.response.model": "claude-opus-5",
"gen_ai.usage.input_tokens": 18420,
"gen_ai.usage.cache_read.input_tokens": 16800,
"gen_ai.usage.output_tokens": 612,
"gen_ai.usage.total_cost": 0.0431,
"gen_ai.response.time_to_first_token": 843000000,
"gen_ai.response.finish_reasons": ["stop"]
}Same record structure for Cursor, Codex, Qoder — field names and structure identical. Different tools enter the same report for cross-tool comparison without per-agent parsing logic. The real organizational measurement barrier was never whether logs exist, but whether different tools' logs can normalize to the same field semantics.
Privacy Control — Two Independent Layers: Collection Scope & Output Masking
Both layers can be baked into the install package at generation time and delivered to every machine.
Layer 1: Collection Scope . --agents enables collection by agent whitelist; unlisted agents on the machine are ignored. --all-agents enables full collection. Decides "which tools enter collection scope".
Layer 2: Output Masking . --mask-mode supports none / all / custom. Under custom, --mask-types specifies types: apiKey, idCard, phone, email, ipAddress, bankCard. Masking scans fields before output and replaces high-confidence sensitive info. In config: mask.mode and mask.types.
{
"mask": { "mode": "custom", "types": ["apiKey", "idCard", "phone"] },
"agents": {
"claude-code": { "enabled": true },
"cursor": { "enabled": true }
}
}What to collect or not stays in the organization's hands, fixed at delivery time, not relying on developer diligence.
Local Operations — CLI Management, Config Rollback, On-Machine Dashboard
Daily ops use intuitive commands; config supports rollback for one-click recovery:
loongsuite-pilot start / stop / restart / status / info / token-usage / rollbackAfter collector starts, local dashboard ready at default port 8765: http://127.0.0.1:8765/ shows real-time token usage, session counts, request and tool-call distributions, and activity by agent, provider, repo — no extra commands. On macOS, Pilot auto-resides in menu bar for at-a-glance status without opening dashboard.
Automated Reach — Qoder Plugin's Silent Install, Self-Healing & Auto-Update
With a solid collection foundation, the remaining challenge is coverage. This section details the distribution control implementation. loongsuite-pilot-installer is a Qoder plugin hooking into SessionStart (session startup). Developers need not know Pilot exists or run any command; just open Qoder and start coding. The plugin auto-decides action:
Installed + config unchanged + process alive : Skip directly (milliseconds)
Installed + config unchanged + process dead : start to revive, no reinstall (seconds)
Not installed / config changed : Background install or overwrite install (minutes, async)
Three branches map to three designs covering the four bottlenecks.
Decision/Execution Separation — Millisecond Detection, Heavy Work to Background
Hook does three cheap checks: executable exists, parameter fingerprint matches, read pidfile + kill -0 to probe process alive. No Node process spawned, so fast-path overhead negligible — session doesn't lag.
Real install is heavy (minutes). Plugin detaches it as a background process detached from CLI process tree: hook returns immediately with "installing in background", session proceeds normally, closing window won't interrupt install. Multiple concurrent sessions use a directory lock with PID liveness check to ensure only one install runs — liveness check, not TTL expiry, because a slow install still alive would be incorrectly reclaimed by TTL, spawning a second install racing to write config.
Bundled Node.js Runtime — Isolated from Developer Environment
Installer downloads a managed Node.js v22 runtime, verifies and pins it in Pilot's own data directory (falls back to system Node only if platform unsupported or download fails). Collection runs independent of developer's Node version or nvm — developer upgrades, switches, or deletes their Node, collection unaffected; conversely, collector never pollutes dev environment. This directly eliminates the most common cause of "install fails".
Parameter Fingerprint — Idempotent Overwrite Install for Config Changes
Fingerprint = SHA-256 of installer URL + all install params, stored in data directory. Admin changes params, increments plugin version, re-delivers; next user session fingerprint mismatches, background overwrite installs with new config, writes new fingerprint; subsequent sessions fingerprint matches, back to millisecond skip.
Overwrite install uses merge semantics for config: omitted params retain old values, local logs and collection checkpoints untouched. First delivery on a machine with manually installed Pilot (no fingerprint) treated as config mismatch — reinstall overwrites, ensuring final config follows admin delivery, not who installed first.
Auto-Update — Version Manifest Comparison & In-Place Restart
Config changes via fingerprint; version upgrades handled by collector itself, no admin re-delivery of plugin. Install writes update channel into config; collector periodically pulls manifest from channel (default ~1 min), compares with local VERSION: if manifest higher, downloads new package, verifies SHA-256, extracts to independent version directory, confirms entry point and deps ready, then in-place restarts collector to new version. Manifest includes a hotfix version bit for emergency gradual rollout.
Zero perception for developers; for admins, collector version convergence doesn't consume plugin delivery cadence — plugin version only bumped on param changes.
User Identity — Data Attributable to People
Org measurement needs "by person" dimension, so events must carry user identity. Plugin at session start reads current Qoder logged-in user's uid from Qoder runtime context, writes into collection config, lands in event's user.id field.
Fully admin-controllable: env var LOONGSUITE_PILOT_USER_ID overrides with org's own identity system; if not provided, collector falls back to hostname — data attributable only to machine, not person.
Organizational Delivery — Install Package Generation & Scoped Distribution
Admin's entire work happens in two consoles: first generate install package with params in Access Center, then upload and distribute via Qoder Enterprise Marketplace. No config files written, no developer contacted.
Step 1: Generate Install Package in Access Center
Enter Access Center, Agent select Qoder or Qoder CN , integration protocol choose "LoongSuite Pilot Collector Plugin Install Package" — generates and downloads collector plugin install package per params. Params match collection capabilities:
Connection Method : Public or Alibaba Cloud intranet
App Name Prefix : Prefix for reported data app name, e.g. ai-coding-agent Trace Integration : Enable, then fill Trace LicenseKey
Log Integration : Enable, choose log write method (e.g. auth-free WebTracking), init log resources
Log Auto-Masking : Maps to Pilot's output masking; enable for sensitive envs
Click "Generate Install Package" to download loongsuite-pilot-installer zip. Same params within validity reuse existing package; "Regenerate" for overwrite — this is the parameter fingerprint mechanism on console side: params unchanged, package unchanged, machines naturally avoid redundant reinstalls.
Step 2: Upload to Enterprise Marketplace
Open Enterprise Marketplace. In Qoder CN console go to "Organization → Extensions & Marketplace → Enterprise Marketplace".
Upload Install Package. Select collection, click "+ Add Content", drag the generated loongsuite-pilot-installer zip into upload area — each top-level folder in zip parsed as a plugin; console lists parsed result, confirm and click "Import".
Confirm Upload Result. Saved content appears in right-side table; plugin upload complete.
Step 3: Configure Distribution Policy
Newly imported content defaults to "Not Distributed" — won't push to anyone; no accidental reach before this step. In content table click "Action → Configure", open "Configure Distribution Policy", choose distribution scope and client strategy, execute distribution.
Distribution scope decides "push to whom"; client strategy decides "what member perceives". Two dimensions combine into a complete rollout rhythm:
Self-Service : Distribution Scope = Not distributed; Client Strategy = —; Effect = Content only visible in marketplace; seed users self-install to validate pipeline
Pilot : Distribution Scope = Group distribution; Client Strategy = Client configurable; Effect = Push to selected groups; members can disable on client
Scale-Up : Distribution Scope = All-users distribution; Client Strategy = Force enable; Effect = Push to all users; forced enable, user cannot disable
Baseline : Distribution Scope = All-users distribution; Client Strategy = Silent enable; Effect = Not shown on client; forced enable in background, becomes org infrastructure
From self-service to gradual, full, then baseline — each step controllable. Plugin carries version; content ops retain "Version History" and "Export Current Version"; rollback by switching version, no developer disruption.
Practical tip: marketplace visibility scope (collection-level) and distribution scope (content-level) are independent switches; keep consistent to avoid members receiving content but unable to find it in marketplace.
Step 4: Verify Installation
Member side zero action; next Qoder session auto-installs. To confirm, run loongsuite-pilot status on member machine to view running state, or let data speak in reporting backend.
Summary & Outlook
The two opening prerequisites are each handled by a link component.
Data normalization by LoongSuite Pilot. Auto-discovers machine's agents, integrates via each's hook/plugin/polling mechanism, normalizes 20 agents' raw activity into a unified GenAI-semantic event schema — three-level session structure, token breakdown with cost, first-token latency, normalized provider values — same field semantics enabling cross-tool aggregation. Collection switches and output masking two independent layers; org decides what to collect.
Collection coverage by loongsuite-pilot-installer plus Qoder Enterprise Marketplace. Plugin does millisecond detection at session start; installs silently in background when needed; bundles Node.js runtime avoiding dev env conflicts; liveness probe keeps process running; parameter fingerprint makes config changes idempotent overwrite installs. Admin generates install package in Access Center, configures distribution scope and client strategy once in Marketplace; thereafter no developer involvement needed.
Collection foundation, automated reach, organizational delivery — three layers each solving one problem, chained into "admin configures once, all members auto-effective". "How is enterprise AI coding actually used" can now be answered with data.
Going forward, delivery shouldn't be bound to a single platform. Portability lies in the installer itself — installer.sh supports fully non-interactive parameterized install: collection scope, masking policy, user identity, reporting address all baked into one command. Any channel that can deliver that command to dev machines can become a delivery channel. Two directions under exploration:
Tool-native enterprise extension distribution. Qoder Enterprise Marketplace exemplifies this; peers include Cursor, Trae (VS Code family) enterprise extension distribution, JetBrains IDE Services, and Claude Code/Codex CLI managed config + hooks. Admin never touches terminal but limited by whether tool vendor opens the channel.
Existing enterprise command delivery channels. Terminal management platforms like CloudShell, MDM, Ansible/SaltStack config management — all capable of pushing a command to a scoped machine set; just configure the above parameterized command as a delivery task. No tool-vendor dependency, no agent restriction; any machine under management gets covered.
References
[1] LoongSuite Pilot repository: https://github.com/alibaba/loongsuite-pilot [2] Output Event Schema:
https://github.com/alibaba/loongsuite-pilot/blob/main/docs/zh-CN/output-event-schema.md[3] Agent Config & Collection Strategy:
https://github.com/alibaba/loongsuite-pilot/blob/main/docs/zh-CN/agents.md[4] Data Masking:
https://github.com/alibaba/loongsuite-pilot/blob/main/docs/zh-CN/masking.md[5] New Agent Onboarding Guide:
https://github.com/alibaba/loongsuite-pilot/blob/main/docs/zh-CN/agent-onboarding.md[6] Alibaba Cloud AI App Integration: AI Coding Agent: https://help.aliyun.com/zh/document_detail/3033878.html [7] AI Coding Audit Overview: https://help.aliyun.com/zh/document_detail/3049374.html [8] Qoder Enterprise Marketplace Docs:
https://docs.qoder.com/zh/account/enterprise/marketplaceSigned-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
