Announcing Zep Archive: Records Retention and Right To Be Forgotten

Building a production-ready user-facing app requires complying with data privacy regulations such as California's CCPA and the EU's GDPR. For many applications, retention of records for a period of time is also required. Apps with conversational interfaces powered by LLMs are no different.

Zep Archive is a collection of features available in Zep that support meeting these data governance challenges, including archiving all message histories and executing Right To Be Forgotten requests.

Implementation is fast and simple. If you're using LangChain, adding the ZepMemory class to your chain and creating users automatically enables Zep Archive features.

Archive all user data, including message histories

When using Zep's MemoryStore, all user sessions and messages, including AI responses, are timestamped and archived. Timestamping and Zep's soft-delete and purge capabilities (described in more detail below) allow time-based retention policies to be efficiently implemented.

While periodic time-based record deletion needs to be configured manually today, we are considering how to offer this as a core feature to enterprise customers.

Fulfill Right To Be Forgotten requests

Zep makes it simple to execute Right To Be Forgotten requests. When using the Zep UserStore, all user data can be deleted with a single API call or via the Zep Web UI.

This dataset includes the user and their chat message history texts and derived data such as extracted metadata, intent analysis results, and embeddings used to search over historical conversations.

await client.user.delete("user123");

Retrieve Chat Histories by Date and Search Terms

Zep's MemoryStore search functionality allows for datetime and keyword searches over a user's chat history, supporting many discovery use cases.

const searchPayload = new MemorySearchPayload({ 
  metadata: {
    "start_date": "2022-01-01",
    "end_date": "2022-01-31"
  },
  text: searchText
});

const searchResults = await zepClient.searchMemory(
  sessionID,
  searchPayload
);

Soft-Deletes and Batched Expunge

Zep soft-deletes data before periodically purging it. This applies to all deleted data, including user data deleted via executed Right To Be Forgotten requests. For customers with large user bases, any impact on application performance is mitigated.

The period between purges can easily be configured and defaults to once an hour.

Next Steps

• Zep Quick Start Guide
• Documentation