Turning Idle Hadoop Clusters into Valuable Data-Driven Products and Processes

Big data has become a strategic focus for many companies, yet the debate continues over whether big‑data initiatives should be driven primarily by technology or by business needs. The ultimate goal is to ensure that big data delivers tangible value rather than merely residing in Hadoop clusters without impact.

In domains with clear commercial models—such as search, online advertising, and recommendation systems—technology‑driven approaches dominate. In contrast, for scenarios where the business value is unclear or undiscovered, a business‑driven approach is more effective. If big data remains unused, it becomes a burden rather than an asset.

Effective monetization of big data involves two pillars: Data‑Driven Processes and Data‑Driven Products .

Building Data-Driven Products

Data‑driven products can be classified into data products and service products .

Data Products

When external company workflows need transformation or internal repetitive processes require optimization, enterprises often package big‑data insights into a data product. Examples include Taobao’s "Quantum Constant" tool for sellers and e‑commerce cross‑selling recommendation engines.

Typical models for data products are:

Directly selling raw data.

Processing internal data and then selling the refined dataset.

Purchasing external data (or crawling it) and integrating it to offer higher‑quality services such as targeted advertising or ad‑effectiveness monitoring.

Providing data‑computation capabilities, e.g., recommendation engines similar to those offered by companies like Baidu.

Crowdsourcing data collection to create a data‑trading marketplace.

Mature product types—search, recommendation, and computational advertising—require deep big‑data expertise combined with business and product insight. Emerging or less mature products (e.g., the aforementioned "Quantum Constant" or third‑party micro‑blog marketing platforms) currently focus on meeting basic data demands, with a gradual shift toward technology‑driven value as the field matures.

Service Products

Service‑oriented data products target end‑users (C‑end) who care more about functionality and experience than about internal business processes. The key distinction is whether the product is data‑intensive or labor‑intensive.

For instance, a traditional restaurant service might rely on manual surveys, whereas a data‑driven approach would analyze users’ browsing behavior to infer preferences (taste, location, spending power) and deliver personalized dining recommendations.

Another example is the evolution from a generic portal to personalized news feeds, as seen in platforms like Toutiao, NetEase News, and Zaker. Personalized recommendation is essentially a data‑driven service product.

Refining and Intelligentizing Processes via Data‑Driven Approaches

Big data can enhance virtually every business process, including marketing, customer management, product management, and human‑resource optimization. Specific applications include:

Predicting future product sales to allocate resources more efficiently.

Identifying potential customers for proactive marketing campaigns.

Segmenting existing users based on behavior patterns to guide product improvements.

Analyzing employee stability to anticipate turnover.

These improvements focus on two core metrics: efficiency and effectiveness. As these metrics become more critical, the demand for data mining and predictive modeling increases.

Conclusion

While it is conceivable that future enterprises will have all products and processes driven by big data, significant challenges remain. In the short term, the primary objective for companies is to unlock the inherent value of big data, apply it effectively, and achieve measurable business outcomes.