Why Most Companies Overlook Their Own Big Data Usage

Big Data Adoption Survey Findings

Recent surveys by Dresner Consulting and Datamation reveal that only about 17% of enterprises acknowledge active use of big‑data technologies, while 47% plan to adopt them without a defined timeline. Despite this low deployment rate, 59% of respondents consider big data "critical" for future competitiveness.

Key Technical Challenges

Analyses from NewVantage Partners (2012) show that most organizations do not face petabyte‑scale storage problems. Instead, the dominant challenges are:

Data variety : integrating structured, semi‑structured, and unstructured sources.

Processing speed : delivering low‑latency analytics on high‑velocity streams.

These issues drive the demand for flexible data models and real‑time processing frameworks.

Technology Responses

Vendors have focused on tools that address variety and speed:

NoSQL databases such as MongoDB (document‑oriented) and DataStax Cassandra (wide‑column) provide schema‑less storage and horizontal scalability.

Stream‑processing engines like Apache Spark (with Structured Streaming) enable in‑memory computation on continuous data flows, reducing latency compared to batch‑oriented Hadoop MapReduce.

Traditional relational systems remain important. Oracle, for example, topped the DB‑Engines popularity ranking in 2015, largely because of its mature capabilities for managing well‑structured, row‑based data—exactly the type of “small, tidy” datasets that many big‑data analytics pipelines ingest before further processing.

Industry Perspectives

Experts caution against treating "big data" as a marketing label:

MySQL engineer Justin Swanhart argues that the focus should be on selecting the appropriate database rather than chasing the hype.

Gartner analyst Nick Heudecker notes that the term has migrated into established domains such as advanced analytics, data science, business intelligence, enterprise information management, in‑memory computing, and overall information infrastructure.

MIT Sloan researcher Michael Schrage emphasizes that the deepest impact of predictive analytics comes not from marginally better forecasts but from fundamentally changing how organizations frame problems and opportunities.

In practice, many firms already run big‑data projects under different names (e.g., “real‑time analytics” or “data lake” initiatives). The remaining barrier is cultural: embedding a data‑driven decision‑making process into the corporate DNA.