Zep for Structured Outputs from Chat History (Video Walkthrough)

Next Steps

1. Learn more in the Zep Structured Outputs Guide.
2. Sign up for Zep's Long-term Memory Service for AI Assistants.

Transcript

00:22

Hi there - I'm Daniel, the founder of Zep. Today I'm going to demonstrate how to use Zep's structured data extraction features. Unlike your LLM provider's JSON mode, Zep extracts data from user dialog quickly and accurately, guaranteeing data field formatting and guarding against hallucination.

00:16

We start off by defining a model of the data we'd like to extract. Zep offers many field types for building your model, from text, date, to different number formats, phone, email, and zip code primitives.

00:04

Each field has a description detailing what should be extracted from the dialog.

00:04

The Regex field type is versatile, and can be used to create e-nums,

00:02

and even comma separated value fields.

00:15

Let's save a dialog between a user and an ecommerce bot into Zep's long-term memory store. The user mentions brand preferences, shoe size, their budget, and selects a shoe to purchase.

00:14

Let's review the model again. We'll be extracting data in the dialog for each of the fields. We'll also be telling Zep how many of the most recent messages it should pass to the extractor to populate the model.

00:12

Let's run our first extraction. We're passing in our model, the last n just discussed, and the current date and time in the user's timezone. I'll speak to this in more depth a little later.

00:07

That completed in less than 500 milliseconds when accounting for overhead. Far faster than using an LLM provider such as OpenAI.

00:06

Note that fields where there is no data in the dialog context have not been hallucinated. They're not set.

00:17

If we have the user provide their email address and run the extraction again, the model is updated. As mentioned, Zep guarantees the format of email and similar fields. Note that we've also inferred the user's first name from the email address.

00:29

Let's pause and take a quick look at this merge_models function. Each extraction call returns a new model with data extracted from the LAST N trailing window of messages. We won't be able to extract data from messages prior to this window. If we run extraction on every chat turn, we'll want to merge newly extracted model values with the model from a prior extraction. This little helper function does just this.

00:06

We'll add several additional messages with new information, and watch as the model is populated with this new data.

00:22

Lastly, let's look at Zep's relative and partial date support. I mentioned earlier that we pass in to the extractor the user's current date and time. If the user or assistant mentions a relative or partial date, such as "tomorrow at 1pm", Zep understands this, and provides us with the value in Python Datetime format.

00:12

I've walked you through using Zep for low-latency and accurate data extraction. We used Python for this example. Zep's TypeScript SDK also offers the same functionality.