ThoughtSpot Spotter 3: Migrating Shared Memory Without Losing Production Business Rules
The article explains how ThoughtSpot Spotter 3's conversation-based rule learning and YAML-based shared memory migration can inadvertently overwrite production business rules during deployment, and outlines a safe release process: export production rules as baseline, merge new corrections, validate with dry-run API, and verify both new and existing queries post-deployment.
ThoughtSpot's Spotter 3 enables users with write permissions to correct the Data Agent during conversation, saving those corrections as shared memory rules attached to a data model. These rules cover business conventions (e.g., "best-selling defaults to quantity sold") and query patterns (Recipes) derived from Liveboards. However, the migration API (available from version 26.8.0.cl) uses a full-replacement mechanism: importing a YAML file replaces the target model's entire shared memory. If the export contains only the newly fixed rule, pre-existing production rules — such as "net sales deduct refunds" and "week starts Monday" — are removed.
Field Selection Determines Business-Aligned Results
In ThoughtSpot's retail demo, the model contains both sales (revenue) and quantity purchased (units sold). When asked "which items sell best", Spotter initially chose sales because "selling" linguistically resembles "sales". The query executed and returned a complete ranking, but the business team needed units sold. A high-priced coat may top revenue while a low-margin shirt leads in volume. Even with correct table names and executable SQL, picking the wrong metric deviates from business intent.
Data teams can document field names, descriptions, and synonyms, but a specific convention is still required: in this retail model, "best-selling" without qualification defaults to quantity purchased; only an explicit "by revenue" request should switch to sales. This convention resolves ambiguity while allowing the same model to serve both replenishment and revenue analysis.
Spotter 3 Writes Conversation Corrections into Shared Memory
With conversation learning enabled, authorized users can correct the Agent directly in chat. For explicit business corrections, Spotter saves the convention as a shared rule linked to the current data model; subsequent users of that model inherit the rule. Shared memory also stores query patterns (Recipes) — for example, a validated weekly dashboard's field selection, granularity, and filters can be generated as a Recipe for reuse.
As teams accumulate rules across domains (store, product, finance), a subtle issue arises: ThoughtSpot documentation notes that even if the data model later changes column names or metric definitions, old memories remain attached. Previously correct Recipes may reference deprecated fields. Teams must therefore track not only whether a rule is saved, but whether it still applies to the current model.
YAML Import Uses Replacement, Risking Rule Loss
Consider a simplified release scenario. Production already holds two rules: "net sales deduct refunds" and "week starts Monday". Development adds a third: "best-selling defaults to quantity sold". The development YAML export contains only this new rule. The import API locates the target model via datamodel_sources and replaces its shared memory with the file's content. Consequently, the two production rules are deleted and only the new rule is written.
This design suits initializing a new model or migrating an entire model's memory to another environment. For an active production model, the release file must contain the complete set of rules that should exist post-deployment. Development and production rules diverge over time — finance may correct refund logic in production while development fixes sorting in test. Both sides accumulate via conversation learning, so their rule sets must be merged before release.
Even a successful new rule does not guarantee a correct release. The stakeholder who requested the fix re-asks "recent best-selling items" and sees the correct unit-based ranking, assuming success. But another user asking "this week's net sales" may now lose the refund deduction and Monday-start conventions. The Agent's behavior then depends on whatever remaining definitions exist in the model. Determining which queries are affected requires actual validation.
Pre-Release Rule Diff, Post-Release Query Verification
To safely deploy the sorting rule, start with production's current shared memory as baseline: export it, then merge the validated development rules. The final YAML should retain all three conventions. Review the diff — only the sorting rule is new; refund and week-start rules are preserved.
Cross-environment checks are also needed: development and production may each have a "Retail Sales" model, but their identifiers and field configurations can differ. Confirm the model ID and referenced columns before import.
ThoughtSpot's migration API offers a dry_run mode that returns the target model's current rule count, projected deletions, and projected insertions. In the scenario, submitting only the new rule shows two deletions and one insertion; submitting all three shows replacements but with the original two re-written.
Automation must not rely solely on HTTP status codes. The API can return VALIDATION_FAILED alongside HTTP 200, indicating the request was received but the file failed validation. Release scripts must read the response body's status field.
Even a passing dry_run does not guarantee business correctness. It validates file structure and model resolution, not whether referenced fields exist in production or whether calculations still hold. For instance, development may identify "refunded" via a flag column, while production requires deriving it from order status. The YAML imports cleanly, yet the refund logic is wrong.
Final acceptance requires live query testing. After deployment, start a new conversation and ask "best-selling products" using a sample where revenue and unit rankings diverge, confirming Spotter uses quantity purchased. Then ask "this week's net sales" using cross-week data with refunded orders, verifying week-start and refund deduction. Where possible, inspect the actual fields, filters, and results the Agent uses.
Test data must expose semantic differences. If revenue and unit rankings coincidentally match, or the test week has no refunds, an incorrect rule can still produce seemingly correct results. Queries that passed in development must be re-validated in production; previously working queries cannot be skipped just because they were untouched in this release.
The original goal was only to fix "best-selling" sorting. Under a full-replacement import, however, the release file determines the final rule set. Adding one correction means the refund definition and week-start convention must also be included in the deployment package. The sorting fix is verified, and the finance colleague's "this week's net sales" query still returns results matching the original business convention. Both questions must pass to confirm the change did not carry away old rules.
Reference sources: ThoughtSpot official case | Why Spotter picks the wrong metric column; ThoughtSpot official docs | Generating shared memory from conversation; ThoughtSpot official docs | Spotter memory and query patterns; ThoughtSpot developer docs | Spotter memory migration API; ThoughtSpot official docs | Cross-environment shared memory migration.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
