Zep v0.17 - Summary Search and Enrichment, Scalability Improvements

Zep 0.17.0, released today, now includes the ability to search over summaries created by Zep, offering a new and powerful path to grounding your LLM apps with long-term memory.

Zep periodically generates summaries from chat histories persisted to the Zep service. A summary and a set of recent chat messages are returned when a developer retrieves recent memory from Zep.

Before v0.17, individual chat messages could be searched and populated into a prompt. This has limitations, as vector similarity search can have poor results when chat messages are short or lack context. Think of a single one-word yes in answer to a question. This is important information but may not result in a vector search result.

Zep stores all historical summaries of a conversation. Now, you can search over these summaries and populate your prompt with them. Since a summary is a distillation of a conversation, they're more likely to match your search query and offer rich, succinct context to the LLM.

import {
  MemorySearchPayload,
  NotFoundError,
  ZepClient
} from '@getzep/zep-js'

const searchPayload = new MemorySearchPayload({
  text: searchText,
  search_scope: 'summary',
  search_type: 'mmr',
  mmr_lambda: 0.5
})
const searchResults = await client.memory.searchMemory(
  sessionID,
  searchPayload,
  3
)
searchResults.forEach((searchResult) => {
  console.debug('Search Result: ', JSON.stringify(searchResult.summary?.content))
  console.debug(
    'Search Result Distance: ',
    JSON.stringify(searchResult.dist)
  )
})

Zep's native support for Maximum Marginal Relevance Re-Ranking is particularly useful with summaries. Successive summaries may include similar content, with Zep's similarity search returning the highest matching results but with little diversity.

MMR re-ranks the results to ensure that the summaries you populate into your prompt are both relevant and each offers additional information to the LLM.

Summary Enrichment with Named Entities

Zep now enriches summaries with Named Entities, similar to the enrichment already offered for individual chat messages.

Search over summaries can be filtered by the named entity metadata added to the summary by Zep.

const searchPayload = new MemorySearchPayload({
   metadata: {
      where: {
         and: [
            {
               jsonpath:
                  '$.system.entities[*] ? (@.Label == "WORK_OF_ART")',
            },
            {
               jsonpath:
                  '$.system.entities[*] ? (@.Name like_regex "^parable*" flag "i")',
            },
         ],
      },
   },
   text: searchText,
});

Scalability Improvements

The Zep service is now entirely stateless, allowing simple scale-out of Zep instances. All background operations are now persisted in a message queue in Zep's back-end Postgres service and executed by Zep instances when available.

Upgrading to v0.17.0

1. To upgrade Zep, pull the latest containers from Zep GitHub Package Repo. If you're using docker compose, you can run:

git pull
docker compose pull

2. Modify your configuration by adding the entities and embeddings sections under summarizer according to your setup.

extractors:
  messages:
    summarizer:
      enabled: true
      entities:
        enabled: true
      embeddings:
        enabled: true
        dimensions: 384
        service: "local"

Alternatively, add the following variables to your environment:

ZEP_EXTRACTORS_MESSAGES_SUMMARIZER_ENTITIES_ENABLED=true
ZEP_EXTRACTORS_MESSAGES_SUMMARIZER_EMBEDDINGS_ENABLED=true
ZEP_EXTRACTORS_MESSAGES_SUMMARIZER_EMBEDDINGS_SERVICE=openai
ZEP_EXTRACTORS_MESSAGES_SUMMARIZER_EMBEDDINGS_DIMENSIONS=1536

Modify the above embedding settings according to your requirements.

3. Upgrade Python or TypeScript SDKs to the latest available versions.

Next Steps

• Zep Quick Start Guide