Secure HDFS with Guardian 5.0: Complete Permission and Quota Guide

Introduction

Hadoop clusters face serious security risks; while MongoDB incidents involve tens of thousands of servers, a relatively small Hadoop deployment can leak over 5,000 TB via HDFS, highlighting the urgency of securing Hadoop, especially HDFS.

Guardian 5.0 Overview

Guardian 5.0 extends existing security solutions with user authentication, authorization, quota management, and resource control. It supports LDAP and Kerberos for authentication, role‑based access control (ARBAC) for authorization, and enforces quotas at the service level.

The platform replaces the traditional OpenLDAP+Kerberos stack with an improved Guardian Directory Service that unifies LDAP and Kerberos, improving authentication efficiency.

Guardian Server implements a full ARBAC model, offers REST APIs, a user‑friendly Web UI, password policies, and JWT‑based SSO, exposing LDAP, REST, and LoginService interfaces for third‑party integration.

Plugins on the third layer provide authentication, authorization, group mapping, and quota management for all TDH components, enabling a single user‑group‑permission model across Hadoop.

HDFS Permission Management

Different Hadoop components use different permission models (POSIX‑like ACL for HDFS, RBAC for Hive, group‑based RBAC for HBase), making cross‑component administration cumbersome.

Guardian 5.0 introduces an enhanced ARBAC model that unifies permission management across components, allowing administrators to use a single Web UI or REST API for granting permissions, while still supporting SQL and HBase shell authorizations.

Predefined HDFS Permissions

READ – read access

WRITE – write access

EXEC – execute access

ADMIN – allows the holder to grant or modify other users’ permissions on the directory

ACCESS – service‑level permission; required for any user to access HDFS services

Administrators can navigate to the “Permissions” menu, select an HDFS service (e.g., hdfs1), and view or edit global and directory‑specific permissions. The UI allows searching resources, editing permissions for users, groups, or roles, and supports batch operations on multiple directories.

Examples illustrate granting READ to user alice on /training, assigning ADMIN on /user/alice, and using the “Add Permission” button to assign roles or groups to any path.

HDFS Quota Configuration

Guardian 5.0 also provides a quota management UI that visualizes each HDFS directory’s allocated space and maximum file count, enabling administrators to modify or delete quotas directly from the “Quota” tab.

Administrators can search for a directory (e.g., health-check-dir), edit its quota, or add new quotas for multiple directories simultaneously using the “Add Quota” button.

Conclusion

Guardian 5.0 transforms traditional command‑line permission and quota configuration into an intuitive graphical interface, simplifying big‑data security enforcement. The platform will continue evolving to deliver a comprehensive, easy‑to‑use security solution for enterprise Hadoop deployments.