Essential 2020 Backend Tech Stack: 14 Categories of Tools and Frameworks

1. Message Queues

For large distributed systems, asynchronous messaging is essential. Kafka provides the highest throughput, especially for big‑data workloads. RocketMQ and RabbitMQ are telecom‑grade and widely used. ActiveMQ is older and less common. Pulsar addresses some Kafka limitations but has a younger ecosystem. For IoT, MQTT (e.g., VerneMQ) is recommended.

2. Caching

Use Caffeine for in‑process caching (default in Spring Boot) due to its speed. For distributed caching, Redis cluster is the primary choice, but be aware of its single‑threaded nature; avoid heavy operations. For large binary data, Memcached can be more suitable. JetCache offers a unified Java API for both local and distributed caches.

3. Sharding (Database Partitioning)

Sharding‑JDBC (now an Apache project) is the recommended driver‑level solution; Mycat is an alternative proxy layer. For teams without dedicated ops, Sharding‑JDBC is easier to adopt. Spring's dynamic datasource works for simple cases. MySQL replication can handle basic read/write separation.

4. Data Synchronization

Canal is the most popular MySQL binlog sync tool in China; Maxwel and similar tools also exist. These can push changes to message queues for downstream processing. General ETL tools (Gobblin, DataX, Logstash, Sqoop) follow a source‑task‑sink pattern, handling both incremental and full data loads.

5. Communication

HTTP + JSON is the easiest to debug; for high performance, binary protocols are preferable. Netty is the de‑facto Java network framework, with OkHttp for lightweight HTTP. For RPC, Thrift, Dubbo, gRPC (using Protobuf) are binary‑oriented; Feign and Hessian work over HTTP. Choose based on latency tolerance and ecosystem.

6. Micro‑services

Recommended stack: Consul for service registry, Nginx + Gateway for API gateway, Apollo for configuration management, SkyWalking for tracing, and Resilience4j (or Sentinel) for circuit breaking. Spring Cloud still uses Eureka, but Consul is now preferred; Nacos and Zookeeper are viable alternatives.

7. Distributed Coordination Tools

Zookeeper, etcd, and Consul provide strong consistency for configuration, distributed locks, naming, and leader election. For distributed transactions, Alibaba's Fescar (now Seata) can be used, though flexible eventual consistency is often sufficient.

8. Monitoring Systems

Prometheus + Grafana + Telegraf is the most popular stack. For log collection, ELKB (Elasticsearch‑Logstash‑Kibana) is common; Loki can be used for lightweight logs. Zabbix works well for small‑scale host monitoring. InfluxDB + Telegraf is a solid time‑series solution.

9. Scheduling

XXL‑Job is a lightweight, Chinese‑origin scheduler with a friendly UI. Quartz is older and uses DB locks for distributed scheduling. Elastic‑Job‑Cloud offers more features but higher operational complexity.

10. Entry Tools (Load Balancing)

HAProxy, LVS, and Keepalived are widely used for traffic entry. CentOS with Ansible is a common server‑side setup.

11. Storage & Search

Elasticsearch and Solr are the leading search engines; Solr is more mature, Elasticsearch offers real‑time capabilities. For columnar storage, HBase (Hadoop) is prevalent; LevelDB is used as an embedded engine. TiDB provides MySQL compatibility with distributed features. Time‑series databases include OpenTSDB, Druid, and Kudu.

12. CI/CD & Containerization

Jenkins remains the primary CI server; TeamCity offers a polished UI. Docker is the base container platform; GitLab CI provides integrated pipelines. Harbor adds security and governance to Docker registries. Kubernetes (with Rancher) is the de‑facto orchestration layer for large‑scale deployments.

13. Problem Diagnosis

For Java OOM analysis, use jmap to dump the heap and MAT for deep inspection. In production, Arthas and perf are useful for real‑time profiling. Linux toolchains complement these utilities.

14. Local Development Tools

Common Java libraries include Druid (most used connection pool in China), HikariCP (fastest), DBCP, and C3P0. For JSON, FastJSON dominates domestically, while Jackson is preferred internationally. Guava remains the go‑to utility library.