Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Limitations, and Practical Guidance

When discussing database scaling, many engineers wonder whether to adopt a middleware‑plus‑sharding approach or a NewSQL distributed database. This article aims to objectively compare the two paradigms by analysing their key characteristics, implementation principles, advantages, drawbacks, and suitable scenarios.

What makes NewSQL advanced? According to the SIGMOD paper "pavlo‑newsql-sigmodrec", Spanner, TiDB, and OceanBase belong to the first‑generation NewSQL architecture, while middleware solutions such as Sharding‑Sphere, Mycat, and DRDS belong to the second generation. Middleware‑based sharding still distributes storage and can achieve horizontal scaling, but it duplicates SQL parsing and execution‑plan generation across the middleware and the underlying DB, leading to inefficiency.

The article presents a simple architectural diagram (omitted here) that highlights several points often promoted by NewSQL vendors:

Traditional DBs are disk‑oriented; NewSQLs use memory‑centric designs for higher efficiency.

Middleware repeats SQL parsing and optimization, reducing performance.

NewSQL distributed transactions are optimized beyond XA, offering higher throughput.

NewSQL stores data using Paxos or Raft multi‑replica protocols, providing true high‑availability and low RTO/RPO.

Automatic sharding, migration, and scaling are transparent to applications, eliminating manual key‑based routing.

While these claims sound attractive, the article scrutinises each point and provides the author’s interpretation.

Distributed Transactions

NewSQL databases still rely on two‑phase commit (2PC)‑like protocols (e.g., Google Percolator’s TSO + MVCC + Snapshot Isolation). Although they reduce lock contention and move part of the commit asynchronously, they inherit the overhead of GID acquisition, network latency, and log persistence, especially when many nodes participate.

SI (optimistic locking) may cause many commit failures under hotspot workloads and offers isolation different from true Repeatable Read.

Strong‑consistency transactions impose significant performance costs; the author suggests considering flexible (BASE) transaction models such as Saga, TCC, or reliable messaging for high‑throughput OLTP scenarios.

HA and Multi‑Region Active‑Active

Traditional master‑slave replication (even semi‑synchronous) can lose data under failure. Modern NewSQLs adopt Paxos/Raft multi‑replica storage, enabling automatic leader election and fast failover, which is now a standard feature of many NewSQL products.

However, true active‑active across distant data centers is limited by network latency; high‑latency links make consensus rounds costly for OLTP workloads.

Horizontal Scaling and Sharding Mechanisms

NewSQLs embed automatic sharding (e.g., TiDB’s region split at 64 MiB) and can rebalance hot spots without DBA intervention. In contrast, middleware‑based sharding requires careful upfront design of split keys, routing rules, and manual expansion procedures.

Online expansion for sharding is possible via asynchronous replication and read‑only switchover, but it still needs coordinated middleware and DB support.

Because NewSQLs use a uniform sharding strategy, some domain‑specific partitioning may be sub‑optimal, leading to distributed transactions for certain business models (e.g., banking customers).

Distributed SQL Support

Both approaches support single‑shard SQL well. NewSQLs, being general‑purpose databases, also support cross‑shard joins, aggregations, and complex queries thanks to built‑in statistics and cost‑based optimizers. Middleware solutions often rely on rule‑based optimization and may lack full cross‑shard capabilities.

NewSQLs typically expose MySQL or PostgreSQL wire protocols, while middleware can route many different DB protocols.

Storage Engine

Traditional RDBMS use B‑Tree engines optimized for disk access; NewSQLs often adopt LSM‑tree engines that turn random writes into sequential writes, improving write throughput at the expense of more complex reads. Additional techniques (SSD, bloom filters, caching) mitigate read penalties.

Maturity and Ecosystem

NewSQLs are still evolving, with strong adoption in internet companies but limited penetration in highly regulated industries like banking. Traditional RDBMS have decades of stability, extensive tooling, and a large pool of DBA talent.

Choosing between the two depends on factors such as the need for strong consistency, data growth predictability, scaling frequency, throughput vs latency priorities, application transparency, and the availability of skilled DBAs.

Conclusion

If several of the listed evaluation questions (e.g., need for strong consistency, unpredictable data growth, frequent scaling, throughput focus, transparent migration) are affirmative, a NewSQL solution may be worth the learning curve. Otherwise, middleware‑plus‑sharding remains a lower‑risk, lower‑cost option that leverages existing relational ecosystems while still meeting most OLTP requirements.