Exploring JD's Big Data Security and Distributed Permission System: Architecture, Principles, and Practices

JD employs over 6,000 data analysts and more than one million data tables, resulting in billions of permission policies that must respond within microseconds while ensuring compliant, efficient data access.

Overall Planning of JD Big Data Security Center

The security center is built on clear security principles (responsibility, compliance, quality, data minimization, least‑privilege, auditability, etc.) and defines roles (security managers, executors, auditors) and corresponding policies. It implements data classification (L1‑L4), fine‑grained permission management, comprehensive auditing, risk compliance, and a security foundation covering data masking, encryption, metadata, cross‑domain sharing, lifecycle management, quality monitoring, and endpoint security.

Data Flow Security Framework

The framework controls security policies across the data lifecycle—collection, storage, processing, distribution, and deletion—by applying appropriate strategies for each stage, such as source compliance, minimal collection, classification, quality assurance, isolation, access control, encryption, and audit.

JD Distributed Permission System

The system addresses three layers of challenges: (1) meeting national compliance (e.g., Level‑4 security, PCI) and diverse business isolation requirements; (2) supporting billions of permission rules for 6,000+ users and millions of tables with high performance; (3) integrating with Hadoop, HBase, Kafka and future components.

Core Modules

1. Subject : Users, user groups, and roles provide a simple ACL model, allowing fast permission requests without dealing with underlying distributed complexities.

2. Resource : Represents databases, tables, files, etc., with multi‑level isolation (market, business line, production account) to satisfy flexible organizational needs.

3. Policy Language : Describes permission objects, resources, and actions, linking subjects and resources.

4. Access Control : High‑performance distributed controller evaluates billions of policies in microseconds across clusters and data centers.

5. Product Architecture : A unified portal enables one‑stop permission request, user/authorization management, and audit/risk control, meeting both business and technical requirements.

In summary, JD's big‑data security center and distributed permission system provide a comprehensive, compliant, and scalable solution for protecting data throughout its lifecycle while supporting massive concurrent access and diverse business scenarios.