Introduction to SPEC and TPC Benchmark Standards for Server Performance Evaluation

In the 1990s the Transaction Processing Performance Council (TPC) was established, introducing the tpmC metric to evaluate online transaction processing capacity by measuring how many new orders a system can handle per minute, primarily for MIS and ERP workloads.

The Standard Performance Evaluation Corporation (SPEC), a global third‑party organization, defines and maintains a series of application performance benchmarks, including SPECcpu2000/2006 for CPU, SPECweb2005 for web servers, SPEC HPC2002 and SPEC MPI2006 for high‑performance computing, and SPECjAppServer2004/SPECjbb2005 for Java applications.

Four major application benchmark families are highlighted: high‑performance computing (HPC) with Linpack, online transaction processing (OLTP) with TPC‑C, web services with SPECweb2005/TPC‑W, and Java application servers with SPECjbb2005, along with vendor‑specific tests for Oracle and SAP.

SPEC, founded in 1988 by several computer equipment vendors, provides widely recognized performance and cost indicators for servers and is used by finance, telecom, and other critical industries as a key selection metric.

SPEC CPU tests are divided into base (basic) and peak (optimized) suites; each further splits into speed tests (single‑threaded execution time) and rate tests (throughput or concurrent performance), with results such as SPECint_base2006, SPECfp_rate_base2006, SPECint®2006, and SPECint®_rate2006.

The SPECsfs2008 benchmark evaluates file‑service performance for NAS systems, measuring throughput and response time, and is used by many NAS manufacturers to validate high‑performance storage solutions.

To obtain official SPEC results, users log into the SPEC/OSG Result Search Engine, select the desired test configuration, optionally filter by hardware vendor, CPU model, or release date, and then fetch the results, which can be downloaded in various formats.

TPC develops business‑application benchmark specifications (e.g., TPC‑C, TPC‑H, TPC‑DS) without providing source code; participants must submit full disclosure reports detailing system configuration, pricing, and maintenance costs for verification.

TPC‑C evaluates performance using the tpmC metric, which is proportional to SPECint_rate_base values and CPU count; the cost‑performance ratio is often expressed as $/tpmC, reflecting system price divided by transaction throughput.

The article concludes with a promotional note directing readers to a 190‑page e‑book titled “Data Center Server Knowledge Complete Guide,” which compiles extensive server‑related topics ranging from processor architectures to storage, networking, and security.