Choosing the Right Database: From RDBMS to NoSQL, NewSQL, and Hadoop

Although hierarchical databases are still widely used on mainframes, relational databases (RDBMS) (SQL) have taken over the database market and perform quite well. The money we store does not go to someone else’s account, we can book a dedicated seat on a plane, and we are not blamed for actions we never performed. Relational databases ensure data integrity by adhering to the ACID (Atomicity, Consistency, Isolation, Durability) principles, a technology that dates back to the 1970s.

So, what has changed? Web technologies have sparked a transformation. Today, many people shop on Amazon, but relational databases were not designed to handle the massive per‑second transactions of such platforms. Their primary limitation lies in the relational mechanism itself.

NoSQL databases offer an alternative mechanism, but this weakens the ACID guarantees. Some NoSQL vendors have made great progress by adopting eventual consistency. As for NewSQL , the idea is to use modern programming languages and technologies to build a relational database without its traditional drawbacks—an approach many NewSQL vendors have taken, with some creating enhanced MySQL solutions.

Hadoop is a completely different species. It is essentially a distributed file system rather than a database, rooted in internet search engine technology. Although Hadoop and its ecosystem (HBase, MapReduce, Hive, Pig, Zookeeper) have turned it into a powerful data platform, it remains a fault‑tolerant, scalable, low‑cost distributed file system whose batch‑processing capabilities are suited for data analysis.

Consider a video‑game company that has operated for ten years, recently launched its hottest game, and shipped products worldwide. Customer information resides in a SQL Server database, which has performed well—until online gameplay caused the database to lag, leading to a poor player experience. Rapid user growth forced the company to spend heavily on additional hardware and software without solving the problem, and losing customers became the greatest risk.

The company decided to split its online user base and run the game on both NoSQL and NewSQL solutions. The IT department chose NoSQL Couchbase (a document‑oriented system similar to MongoDB) and NewSQL VoltDB .

Couchbase is open source, includes an integrated caching mechanism, and automatically propagates data across multiple nodes. VoltDB follows ACID principles, is fault‑tolerant, horizontally scalable, and features a shared‑nothing in‑memory architecture. Both systems can operate together; a detailed comparison would require testing, benchmarking, and deep analysis.

With online operations running smoothly, the company now wants to analyze its data to identify the best markets for expansion. To determine which country to target, it must merge user data from the SQL Server data warehouse with the online game database, then run analytical reports. This is where Hadoop reappears: a Hadoop cluster merges the two data sources, and the open‑source R language, together with its MapReduce module, generates the final analysis report.