Mastering Apache Ranger: Architecture, Workflow, and Batch Policy Automation

Apache Ranger, meaning “park ranger”, serves as a centralized security management framework for the Hadoop ecosystem, enabling fine‑grained access control for components such as HDFS, Hive, HBase and YARN.

Key Features

Unified web UI and REST API for managing security policies.

Fine‑grained control over Hadoop component operations.

Standardized authorization mechanisms.

Support for role‑based and attribute‑based access control.

Centralized audit of user and admin actions.

Architecture

Ranger consists of three main components:

Ranger Admin : core module with a web console and REST endpoints for policy definition.

Agent Plugin : embedded in Hadoop services, periodically pulls policies from the Admin and enforces them while logging audit data.

User Sync : synchronizes OS users and groups into Ranger’s database.

Workflow

Administrators create or modify policies via the Ranger Admin UI. Agent Plugins poll the Admin (default every 30 seconds) to retrieve the latest policies, cache them locally, and enforce authorization when a user requests data from a Hadoop service. Policy changes are applied automatically after the next poll.

For Hive, the plugin uses

org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerFactory

and HiveAuthorizer. A dedicated PolicyRefresher thread refreshes policies into a local JSON cache for fast authorization decisions.

Practical Administration

Using the web UI, administrators can define policies for HDFS paths, Hive tables, columns, and row‑level filters. Example screenshots show a policy granting users /user, /user/rangerpath/, and sub‑directories access without recursion, and audit logs displaying authentication and authorization events.

Ranger also supports temporary policies for short‑term authorizations, masking of sensitive columns (e.g., hashing the lname field in the foodmart.customer table), and row‑level filters that hide specific records.

Batch Policy Management

To avoid manual, error‑prone policy entry, a Java client can call Ranger’s REST API for bulk create, update, delete, and query operations. The core API helper method is:

public ApiResult execRangerApi(String url, String method, String requestBody) {
    HadoopConfig.Ranger ranger = this.hadoop.getRanger();
    String baseUrl = ranger.getApiBaseUrl();
    String user = ranger.getUser();
    String password = ranger.getPassword();
    String fullUrl = baseUrl + url;
    String auth = user + ":" + password;
    String authInfo = DatatypeConverter.printBase64Binary(auth.getBytes());
    HttpRequest request = null;
    if (method.equalsIgnoreCase("GET")) {
        request = HttpRequest.get(fullUrl);
    } else if (method.equalsIgnoreCase("POST")) {
        request = HttpRequest.post(fullUrl);
    } else if (method.equalsIgnoreCase("PUT")) {
        request = HttpRequest.put(fullUrl);
    } else if (method.equalsIgnoreCase("DELETE")) {
        request = HttpRequest.delete(fullUrl);
    }
    ((HttpRequest) ((HttpRequest) ((HttpRequest) request.header("Authorization", "Basic " + authInfo))
            .header("Accept", "application/json"))
            .header("Content-Type", "application/json")
            .header("X-XSRF-HEADER", "valid");
    if (requestBody != null && !requestBody.isEmpty())
        request.body(requestBody);
    HttpResponse response = request.execute();
    ApiResult result = new ApiResult(this);
    result.setHttpCode(response.getStatus());
    result.setBodyRaw(response.body());
    return result;
}

Policy creation and modification use this helper to POST or PUT JSON representations of Policy objects. After building the request, policies can also be submitted via a curl command:

curl -H "Content-Type:application/json" -H "X-Token:token-name" -X POST "http://web-url&appUser=user-name" -d "[\"ranger-policy\"]"

These scripts dramatically improve operational efficiency when handling large numbers of policies.

Reference: Apache Ranger official site – http://ranger.apache.org/