LangchainJS Now Supports Zep!

LangchainJS now ships with Zep Memory and Retriever classes. Upgrade to LangchainJS 0.0.86 or later to take advantage of Zep's long-term memory, auto-summarization, vector search, and named entity extraction.

We'll explore an example Langchain ConversationalChain app below. Note that these are code excerpts and the full example may be found in our Zep by Example repo.

Instantiate a Zep Memory class

SessionIDs uniquely identify your user. We assume you have a Zep server running on localhost user Zep's docker compose setup. Zep also has a quick Render deployment option for development use cases.

import {ZepMemory} from "langchain/memory/zep";

const sessionId = uuidv4();
const zepURL = "http://localhost:8000";

const memory = new ZepMemory({
    sessionId,
    baseURL: zepURL,
});

Load some example messages into Memory

For the purposes of this demo application, we'll load some historical messages into Memory. We do this by writing directly to the Zep server using Zep's zep-js SDK.

const history = [
        {
            "role": "human",
            "content": "Who was Octavia Butler?"
        },
        {
            "role": "ai",
            "content": (
                "Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American" +
                " science fiction author."
            ),
        },
        ...
{
            "role": "ai",
            "content": (
                "Parable of the Sower is a science fiction novel by Octavia Butler," +
                " published in 1993. It follows the story of Lauren Olamina, a young woman" +
                " living in a dystopian future where society has collapsed due to" +
                " environmental disasters, poverty, and violence."
            ),
        },
];

const messages = history.map(
	({role, content}) => new Message({role, content})
);
const zepMemory = new Memory({messages});

await zepClient.addMemory(sessionId, zepMemory);

Create a chain and run it

By utilizing the ZepMemory with the chain, human and AI messages are automatically added to the chat history.

const chain = new ConversationChain({llm: model, memory});

const res1 = await chain.call(
	{input: "Please recommend books similar to Parable of the Sower."}
);
console.log({res1, memory: await memory.loadMemoryVariables({})});

/*
{
  res1: {
    response: "You might enjoy reading The Hunger Games by Suzanne Collins, The Road by Cormac McCarthy, or The Handmaid's Tale by Margaret Atwood."
  },
  memory: {
    history: 'human: What awards did she win?\n' +
      'ai: Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.\n' +
      'human: Which other women sci-fi writers might I want to read?\n' +
      'ai: You might want to read Ursula K. Le Guin or Joanna Russ.\n' +
      "human: Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\n" +
      'ai: Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.\n' +
      'Human: Please recommend books similar to Parable of the Sower.\n' +
      "AI: You might enjoy reading The Hunger Games by Suzanne Collins, The Road by Cormac McCarthy, or The Handmaid's Tale by Margaret Atwood."
  }
}
    */

Use the ZepRetriever to find similar messages

We can find relevant historical messages in long-term memory by using the ZepRetriever class. Retrievers can be used by chains to populate context in a prompt, or as a Tool by agents.

Zep offers semantic search over historical messages using a cosine similarity. As a developer, you don't need to embed messages prior to providing them to Zep. This is done automatically as soon as a message is persisted by the ZepMemory instance.

In the example below,

const retriever = new ZepRetriever({sessionId, url: zepURL})
const relevantResults = await retriever.getRelevantDocuments("I enjoy dystopian sci-fi novels.")
relevantResults.forEach(element => {
        if (element.metadata.dist >= 0.8) {
            console.log(`${element.pageContent} => ${element.metadata.dist}`);
        }
});

/*
You might enjoy reading The Hunger Games by Suzanne Collins, The Road by Cormac McCarthy, or The Handmaid's Tale by Margaret Atwood. => 0.8598901744352312
Which other women sci-fi writers might I want to read? => 0.8353206100397812
You might want to read Ursula K. Le Guin or Joanna Russ. => 0.8236142000294512
Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ. => 0.8089540052202283
Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence. => 0.8052832541478573
Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship. => 0.8038596194093931
*/

Accessing Summaries and Zep Metadata, including Named Entities

Summaries, named entities, and other metadata generateed from memories persisted by Langchain's ZepMemory class can be accessed using Zep's ZepClient.

In the code example below, we first view the summary autogenerated once the number of messages in memory exceed a threshold set by the developer.

const serverMemory = await zepClient.getMemory(sessionId);
console.log(serverMemory?.summary)

/*
 Summary {
  uuid: '0ab0002e-84c1-4d68-ab8b-952bf169b3be',
  created_at: '2023-05-31T01:11:24.792728Z',
  content: "The human asks about Octavia Butler and the AI identifies her as an American science fiction author. The human then inquires about which of her books were made into movies and the AI mentions the FX series Kindred. The human also asks about Butler's contemporaries and the AI names Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.",
  recent_message_uuid: '90bf11fc-359b-456b-afa1-d498964f8212',
  token_count: 393
}
*/

With Zep's EntityExtractor, developers can build sophisticated features that:

• Trigger the use of custom prompts or agent branching;
• Annotate the chat history, enhancing the experience for users with links to additional information, services, or products.
• Evaluate human and agent messages further to extract dates, currencies, people's names, place names, etc.

Alongside the labels and identified entities, the character offsets of the entities are provided. This makes it simple to build chat annotations into your app!

const lastMessage = serverMemory?.messages[serverMemory?.messages.length - 1];
console.log(JSON.stringify(lastMessage));

/*
{
  "uuid": "5af9edbb-6aa8-40f5-917b-4a088bb815e4",
  "created_at": "2023-05-31T01:11:26.71254Z",
  "role": "AI",
  "content": "You might enjoy reading The Hunger Games by Suzanne Collins, The Road by Cormac McCarthy, or The Handmaid's Tale by Margaret Atwood.",
  "token_count": 31,
  "metadata": {
    "system": {
      "entities": [
        {
          "Label": "WORK_OF_ART",
          "Matches": [
            {
              "End": 40,
              "Start": 24,
              "Text": "The Hunger Games"
            }
          ],
          "Name": "The Hunger Games"
        },
        {
          "Label": "PERSON",
          "Matches": [
            {
              "End": 59,
              "Start": 44,
              "Text": "Suzanne Collins"
            }
          ],
          "Name": "Suzanne Collins"
        },
        {
          "Label": "WORK_OF_ART",
          "Matches": [
            {
              "End": 88,
              "Start": 61,
              "Text": "The Road by Cormac McCarthy"
            }
          ],
          "Name": "The Road by Cormac McCarthy"
        },
        {
          "Label": "WORK_OF_ART",
          "Matches": [
            {
              "End": 112,
              "Start": 93,
              "Text": "The Handmaid's Tale"
            }
          ],
          "Name": "The Handmaid's Tale"
        },
        {
          "Label": "PERSON",
          "Matches": [
            {
              "End": 131,
              "Start": 116,
              "Text": "Margaret Atwood"
            }
          ],
          "Name": "Margaret Atwood"
        }
      ]
    }
  }
}
*/

Note that the above are code excerpts and the full example may be found in our Zep by Example repo.

Next steps

• Follow the Zep Quick Start Guide for installation and SDK instructions.