Is Knowing You Too Dangerous? MemPrivacy Secures AI Agent Memory with Local Privacy

In the emerging AI memory era, agents increasingly act like personal assistants that remember habits, schedules, health status, and even build a detailed personal profile. This raises a critical question: if such memories are uploaded to the cloud, can privacy remain safe?

OpenAI Privacy‑Filter

On April 22, OpenAI released the lightweight privacy‑filter model (1.5 B parameters, 50 M activation parameters, bidirectional token‑classification, 128k context) to detect and mask PII. It replaces detected entities with generic tags (e.g., [PRIVATE_PERSON]) or asterisks. While fast, it provides only eight coarse tags, which is insufficient for long‑term agent memory that requires nuanced understanding of context, relationships, and semantics.

MemPrivacy: Local Reversible Pseudonymization

MemTensor, in collaboration with HONOR and Tongji University, open‑sourced the MemPrivacy framework. Its core idea is local reversible pseudonymization : sensitive fragments are replaced on the device with fine‑grained typed placeholders such as <Health_Info_1>, while the true values stay encrypted in a local database.

The pipeline consists of three steps:

Edge‑side upstream sanitization : the device runs a lightweight MemPrivacy model to identify privacy spans and replace them according to user‑defined protection levels.

Cloud‑side secure processing : the cloud model receives only placeholders (e.g., "My blood pressure is <Health_Info_1>.") and can still reason about the type of information.

Edge‑side downstream recovery : after the cloud generates a response, the device restores the original values from the local store before presenting them to the user.

This design ensures that critical data never leaves the device while the cloud retains enough structure to provide personalized services.

Protection Strategies Comparison

Three approaches are compared:

No protection : raw data is uploaded, offering the best personalization but exposing all sensitive information.

Full masking : all privacy content is removed or replaced with asterisks, which eliminates risk but destroys the semantic context needed for personalization.

MemPrivacy’s fine‑grained placeholders : retains semantic types while keeping raw values local, achieving a balance between privacy and utility.

Benchmark Results

MemPrivacy was evaluated on two benchmarks:

MemPrivacy‑Bench (200 users, 155 k privacy items, bilingual).

PersonaMem‑v2 (out‑of‑distribution long‑dialogue set).

On MemPrivacy‑Bench, OpenAI privacy‑filter achieved a composite F1 of 35.50 %, whereas MemPrivacy‑4B‑RL reached 85.97 % – a 50.47 % absolute improvement. Similar gains (~9 % higher) were observed on PersonaMem‑v2. The advantage stems from MemPrivacy’s task‑specific training and fine‑grained type system, not merely model size.

Four‑Level Privacy Taxonomy (PL1‑PL4)

MemPrivacy defines a hierarchical classification:

PL4 – Critical core : passwords, API keys, session tokens; zero‑tolerance blocking.

PL3 – High‑risk sensitive : medical diagnoses, precise location, biometric data; strict protection.

PL2 – Identity anchors : real name, address, phone, email; identifiable but less critical.

PL1 – Basic profile : habits, preferences, non‑diagnostic emotions; safe for long‑term memory.

This taxonomy lets users tune protection thresholds, avoiding the “all‑or‑nothing” approach of traditional filters.

Training Procedure

MemPrivacy models (0.6 B, 1.7 B, 4 B) are built on the Qwen‑3 series. Training proceeds in two stages:

SFT (Supervised Fine‑Tuning) : 26 k high‑quality multi‑turn dialogues teach the model basic privacy span detection, type identification, and placeholder substitution.

GRPO (Reward‑based Policy Optimization) : a structured reward using F1 extraction scores refines the model on ambiguous boundaries (e.g., distinguishing a generic ID from a credential).

This two‑stage regime improves recall‑precision balance on fuzzy, context‑dependent privacy cues.

Utility Loss Evaluation

When traditional irreversible masking was applied, three memory systems suffered accuracy drops of 26.67 %, 41.87 %, and 16.99 %. Under MemPrivacy with all protection levels enabled (PL4+PL3+PL2), utility loss stayed between 0.71 % and 1.60 %; protecting only PL4 caused less than 0.89 % degradation.

Conclusion

MemPrivacy demonstrates that privacy and agent intelligence need not be mutually exclusive. By keeping raw data on‑device and transmitting only typed placeholders, it achieves high‑precision privacy extraction while preserving the semantic utility required for personalized agents. The models, benchmarks, and taxonomy are fully open‑source, offering a practical foundation for the next generation of edge‑cloud AI assistants.