Why Post‑Filtering Fails in Enterprise RAG and How to Securely Pre‑Filter

Interview Scenario that Highlights the Problem

A candidate described an internal knowledge‑base Q&A system that performed department‑based filtering after retrieval, removing unauthorized documents from the Top‑K results. The interviewer pointed out that the vector database had already accessed sensitive documents, which would fail a security audit because data access occurred before permission checks.

Why Post‑Filtering Is Inadequate

Data Access Occurs Early – Vector retrieval reads documents from storage; even if they are not displayed, the access violates the principle of least privilege in regulated environments.

Top‑K Slots Are Wasted – Unauthorized documents occupy the Top‑K results, reducing the number of usable hits and degrading retrieval quality.

Higher Leakage Risk in Multi‑Tenant Setups – Shared vector spaces make documents from different tenants visible to each other during retrieval, which is unacceptable for compliance‑heavy sectors.

The correct approach is pre‑filtering : filter out unauthorized chunks before the similarity computation.

Designing Permission Metadata in Vector Databases

Each chunk stores permission metadata that can be used as filter conditions during search. Common metadata models include:

Department‑level : dept_id = ["finance", "hr"] Role‑level : required_role_level = 3 Document Classification : classification = "confidential" User Whitelist :

allowed_user_ids = ["u001", "u002"]

def insert_chunk_with_permissions(
    collection,
    chunk_text: str,
    chunk_embedding: list,
    doc_id: str,
    permissions: dict
):
    """Insert a document chunk with permission metadata.
    Example permissions:
    {
        "dept_ids": ["finance", "legal"],
        "min_role_level": 2,
        "classification": "internal",
        "allowed_user_ids": []
    }
    """
    collection.insert([{
        "id": generate_chunk_id(),
        "text": chunk_text,
        "embedding": chunk_embedding,
        "doc_id": doc_id,
        "dept_ids": permissions.get("dept_ids", []),
        "min_role_level": permissions.get("min_role_level", 0),
        "classification": permissions.get("classification", "public"),
        "allowed_user_ids": permissions.get("allowed_user_ids", []),
    }])

Vector Database Filter Support

Different vector stores provide varying levels of metadata filtering:

Milvus : Full support with high‑performance index‑level filtering and complex boolean expressions.

Qdrant : Full support via payload filters, tightly integrated with vector search.

Weaviate : Basic support using where filters.

Chroma : Basic equality/in‑list filtering only.

FAISS : No native filter support; requires custom implementation.

Tenant Isolation in Practice

In a financial‑insurance project with 5,000 contract documents, each insurer’s data is stored in a separate tenant. Qdrant was chosen, and a tenant_id payload filter is always supplied to the search request, preventing cross‑tenant access.

Secure Transmission of User Permissions

Pre‑filtering only works if the backend receives trustworthy permission data. Never trust client‑side claims; instead, validate a JWT token on the server and derive filter conditions from the verified payload.

import jwt
from functools import lru_cache

def build_permission_filter(auth_token: str) -> dict:
    """Parse a JWT token and build a vector‑search filter.
    The client‑provided claims are ignored.
    """
    try:
        payload = jwt.decode(auth_token, key=JWT_SECRET_KEY, algorithms=["HS256"])
    except jwt.InvalidTokenError:
        raise PermissionError("Invalid authentication token")

    user_id = payload["sub"]
    role_level = payload.get("role_level", 0)
    # Retrieve the full permission set from a permission service (cached)
    user_permissions = get_user_permissions_cached(user_id)

    permission_filter = {
        "must": [
            {"key": "dept_ids", "match": {"any": user_permissions["accessible_depts"]}},
            {"key": "min_role_level", "range": {"lte": role_level}},
        ]
    }
    return permission_filter

@lru_cache(maxsize=1024)
def get_user_permissions_cached(user_id: str) -> dict:
    """Cache permission look‑ups for 60 seconds to reduce load.
    """
    return permission_service.get_permissions(user_id)

Cache TTL balances performance and freshness; a 60‑second TTL works for most scenarios, while immediate revocation (e.g., employee termination) requires explicit cache invalidation.

Multi‑Tenant RAG Isolation Strategies

Three isolation granularities, increasing in security and cost:

Metadata Filtering (lightest) : All tenants share a collection; a tenant_id field filters results. Simple but vulnerable if the filter is omitted.

Partition Isolation (mid‑range) : Each tenant gets a separate partition within the same collection. Safer than metadata alone, with moderate overhead.

Collection Isolation (strongest) : Each tenant has its own collection, providing full physical separation at the cost of higher resource usage and management complexity.

In the referenced project, a hybrid approach was used: Partition Isolation + Metadata Filtering . Each insurance company occupies a distinct partition, and the tenant_id metadata provides an additional safety net.

Handling Dynamic Permission Changes

Static permissions embedded in chunks are hard to update. Instead, store permission relationships in a relational database and join them at query time.

def search_with_realtime_permissions(
    query: str,
    user_id: str,
    vector_db,
    permission_db,
    top_k: int = 5
) -> list:
    """Search using up‑to‑date permissions.
    Permissions live in a relational DB; chunks only store document IDs.
    """
    accessible_doc_ids = permission_db.get_accessible_docs(user_id)
    if not accessible_doc_ids:
        return []
    results = vector_db.search(
        query_embedding=embed(query),
        filter={"doc_id": {"$in": accessible_doc_ids}},
        limit=top_k,
    )
    return results

Cache the list of accessible_doc_ids for 30‑60 seconds in Redis, and actively invalidate the cache on permission changes or employee termination to guarantee immediate revocation.

Interview Answer Blueprint

When asked about RAG permission control, structure the response as follows:

State the three drawbacks of post‑filtering (30 s).

Explain pre‑filtering implementation, including chunk metadata, JWT verification, and cache design (1 min).

Describe the three tenant‑isolation strategies and the chosen hybrid solution (1 min).

Cover dynamic permission updates with relational‑DB joins and cache invalidation (30 s, bonus points).

Conclusion

Permission control distinguishes a production‑grade enterprise RAG system from a quick demo. Implementing pre‑filtering at the retrieval layer, designing robust metadata, validating tokens server‑side, and planning for dynamic changes are essential to pass security audits and maintain data confidentiality.