Tagged articles

token budgeting

8 articles · Page 1 of 1

Sep 4, 2026 · Artificial Intelligence

AI Bills Skyrocket Despite Cheaper Models: The Agent Cost Multiplier Effect

As model inference prices drop, AI costs surge because Agent architectures multiply model calls per user request; the article breaks down the four-layer cost structure and offers six practical governance tactics—model routing, prompt caching, call-chain slimming, token budgets, observability, and chargebacks—to build a sustainable AI FinOps practice.

AI cost managementFinOpsObservability

0 likes · 15 min read

AI Bills Skyrocket Despite Cheaper Models: The Agent Cost Multiplier Effect

AI Engineer Programming

Jul 23, 2026 · R&D Management

When Companies Mandate AI-Only Coding, Budgets Override Real Productivity

A junior engineer recounts how his Norwegian tech firm first forced developers to code exclusively with AI, then imposed strict token budgets and bonuses, leading senior staff to abandon AI tools, revealing that financial incentives, not actual productivity gains, dominate corporate AI adoption decisions.

AI codingindustry trendsmanagement

0 likes · 7 min read

When Companies Mandate AI-Only Coding, Budgets Override Real Productivity

Ray's Galactic Tech

Jul 16, 2026 · Artificial Intelligence

K8s, Kafka, Nacos Agent Platform to Prevent Token Bankruptcy and Skill Avalanches

The article details how a production‑grade Agent platform built on Kubernetes, Kafka, and Nacos addresses token budget overruns, uncontrolled skill execution, and RAG hallucinations by introducing a four‑layer runtime architecture, token pre‑allocation, explicit state management, dynamic governance policies, and robust skill specifications.

KafkaKubernetesLLM Agents

0 likes · 32 min read

K8s, Kafka, Nacos Agent Platform to Prevent Token Bankruptcy and Skill Avalanches

Xike

Jun 11, 2026 · Artificial Intelligence

Adding Memory: Enabling Multi‑Turn Conversations in an LLM Agent

This guide demonstrates how to replace a simple message list with a ContextManager that tracks user and assistant turns, estimates token usage, applies a sliding‑window truncation based on a token budget, and provides a single build_for_llm entry point to keep multi‑turn dialogues stable and observable.

Context ManagementLLMPython

0 likes · 11 min read

Adding Memory: Enabling Multi‑Turn Conversations in an LLM Agent

Xike

Jun 2, 2026 · Artificial Intelligence

Why Agent Conversations Aren’t Just Chat Logs: Effective Context Management

The article explains that an Agent’s context is a structured snapshot built from role contracts, tool trajectories, and window budgeting, not a raw chat transcript, and details how proper context handling prevents forgetting, token bloat, and tool‑call mismatches in multi‑turn LLM workflows.

Context ManagementLLM AgentsReact

0 likes · 16 min read

Why Agent Conversations Aren’t Just Chat Logs: Effective Context Management

Architect Practice

Jun 1, 2026 · Artificial Intelligence

When AI Rate Limiting Goes Wrong: A Four‑Dimension Framework and Three‑Layer Gateway in Practice

A midnight alarm at a fintech AI platform revealed that traditional QPS throttling missed a runaway Agent that consumed hundreds of times more tokens, prompting a detailed analysis of four token‑based limiting dimensions, three‑layer gateway design, agent‑specific controls, semantic caching, and tool selection to prevent similar “ghost avalanche” failures.

AI rate limitingLLM Operationsagent safety

0 likes · 20 min read

When AI Rate Limiting Goes Wrong: A Four‑Dimension Framework and Three‑Layer Gateway in Practice

Linyb Geek Road

May 12, 2026 · Artificial Intelligence

10 Open‑Source Tools Cutting AI Agent Costs Ten‑Fold: Prompt Compression, Memory Management, Model Routing

The article explains how AI agents become expensive because they ingest massive, irrelevant context and shows ten open‑source projects—LLMLingua, mem0, LiteLLM, LlamaIndex + Chroma, Letta, Guidance, Aider, tiktoken + ttok—that compress prompts, manage memory, route models dynamically, add retrieval‑augmented generation, and enforce token budgeting, collectively reducing daily token usage by millions and slashing costs dramatically.

AI AgentsMemory ManagementRetrieval-Augmented Generation

0 likes · 17 min read

10 Open‑Source Tools Cutting AI Agent Costs Ten‑Fold: Prompt Compression, Memory Management, Model Routing

Wu Shixiong's Large Model Academy

Apr 13, 2026 · Artificial Intelligence

Turning ReAct from Demo to Production: Handling Failures, Loops, and Token Budgets

This article explains how to upgrade a ReAct agent from a proof‑of‑concept to a production‑ready system by classifying tool failures, detecting repeated search loops, managing token budgets, and adding structured logging, complete with Python implementations and practical interview guidance.

Agent EngineeringLLMLoop Detection

0 likes · 24 min read

Turning ReAct from Demo to Production: Handling Failures, Loops, and Token Budgets