From 4 Hours to 1 Hour: Optimizing Coupon Calculations with Storm Stream Processing

In 2014 I worked on the coupon calculation service for eLong’s promotion team, which relied on the Storm stream‑processing framework to compute real‑time discount rules for hotels across cities.

Background

The service used Storm because its core language, Clojure, was uncommon, and many developers were curious about it. As mobile traffic surged, the coupon calculation became a performance bottleneck, taking several hours for a full‑batch run.

Exploring Storm

To understand Storm, I studied its topology: a Spout (data source) feeds streams into Bolts (processing units), similar to water flowing through pipes. I built a simple “Hello World” topology and began investigating its concurrency model.

Identifying the Bottleneck

By breaking the service into three stages—data extraction, Storm computation, and persistence—I examined logs and found that data extraction (pulling hotel information into Redis) consumed over two hours of the four‑hour total.

Refactoring the Extraction Service

The extraction service’s thread model limited each node to two threads. I rewrote it to allow horizontal scaling and made the worker thread count configurable, using insights from RocketMQ’s threading design.

Results

After a week of refactoring and thorough code reviews and testing, the new service ran on three nodes with eight workers each. Full‑batch processing time dropped from four hours to about 1 hour 15 minutes, with the extraction stage now taking only 40 minutes.

Conclusion

The experience taught me the importance of understanding a framework’s architecture, profiling each pipeline stage, and applying systematic refactoring to achieve dramatic performance gains.