I Integrated AI into Our Java System in One Week — The Token Bill Almost Killed Me
A developer shares a week-long journey integrating AI into a Java customer service system, detailing how over-engineering with Spring AI Graph was replaced by a simple ChatClient + Tools approach, token costs were cut 80% via model routing, context compression, and semantic caching, a production latency spike was fixed with Tool result limits and circuit breakers, and three operational essentials — budget alerts, conversation logging, and a kill switch — were added for sustainability.
