How QingStor’s Object Storage Architecture Powers Massive Data Scalability

Core Advantages

QingStor object storage offers a flat structure without directory hierarchies, supports petabyte‑scale data volumes, handles unstructured data of any type, and provides RESTful HTTP APIs for convenient access.

Global Data Model

The system consists of a Global layer composed of multiple Zones (data‑center regions). Each Zone contains many Buckets, and objects are stored within these Buckets without size or quantity limits.

Architecture Overview

The backend is divided into four subsystems: the Gateway (access) service, the Index subsystem for metadata, the Storage subsystem for data entities, and the Event subsystem for asynchronous processing and distributed task scheduling.

Gateway receives requests, parses protocols, and performs read/write operations.

Index stores object metadata using a KV engine with LSM indexing for fast writes and ordered queries.

Storage ensures reliable persistence of data objects.

Event handles lifecycle management, cross‑region replication, and custom callbacks.

Subsystem Implementation

The Gateway service is stateless, enabling horizontal scaling and high availability through DNS‑based virtual IP failover and request‑level failover. It aggregates small‑file writes into sequential appends, reducing random I/O and improving throughput.

The Index subsystem shards data by object name prefixes, uses three‑copy replication for metadata safety, and leverages LSM‑based KV storage for fast writes and efficient list operations.

The Storage subsystem groups data into storage groups (standard or low‑frequency) to separate hot and cold data, allowing flexible hardware choices and fault isolation.

Massive Small‑File Optimization

Small files are merged into large container files to reduce metadata overhead and improve storage utilization. Writes are performed as sequential appends, and concurrent writes are batched into a single I/O operation. Reads use the container file path plus offset to locate data, reusing file handles for efficiency.

Key Features

QingStor provides lifecycle management (automatic deletion or tiering), cross‑region replication for backup and low‑latency access, and a rich set of APIs compatible with S3, along with SDKs for major languages, command‑line tools (qsctl and qscamel), and support for data processing tasks such as video transcoding and image watermarking.

Traffic Industry Best Practice

In intelligent transportation platforms, massive video and image streams from highways and toll stations are stored in QingStor, leveraging its high concurrency, multi‑copy strong consistency, event‑driven processing for plate recognition, and cross‑region replication for near‑real‑time access and disaster recovery.