Session Metadata 👾, Custom OpenAI Endpoints 🔧, & Kubernetes Deployment 🔥

With Zep v0.8.0, released today, developers can associate arbitrary metadata with Sessions. This metadata is persisted in the Zep Memory Store and is available when your user next uses your chat product.

Other significant enhancements include a new Kubernetes deployment option and the ability to use custom OpenAI endpoint APIs.

Session Metadata

With Zep's new Session metadata feature, developers can persist session-related data and later retrieve these alongside chat history related to the session.

Metadata persists for the life of the Session and will only be expunged when the Session is deleted. Keys can however be deleted by persisting an empty or null value.

Alongside a new Session model class, we've added several new client methods add_session and get_session, and their related asyncio aadd_session and aget_session methods. add_session upserts a session, allowing new metadata keys to be added.

  base_url = "http://localhost:8000"  # TODO: Replace with Zep API URL
  api_key = "YOUR_API_KEY"  # TODO: Replace with your API key

  with ZepClient(base_url, api_key) as client:
      session_id = uuid.uuid4().hex

      try:
          session = Session(session_id=session_id, metadata={"foo": "bar"})
          client.add_session(session)
      except APIError as e:
          print(f"Unable to create session {session_id} got error: {e}")

We can use get_session to retrieve a Session. You'll note below that the metadata document includes both the food and bar keys and their values.

  session = Session(session_id=session_id, metadata={"bar": "foo"})
  client.add_session(session)

  session = client.get_session(session_id)
  print(session.dict())

{'uuid': '944045d3-82ea-41c3-9228-13df83960eba', 'created_at': '2023-07-12T16:49:53.00569Z', 'updated_at': '2023-07-12T16:49:53.00569Z', 'deleted_at': '0001-01-01T00:00:00Z', 'session_id': '487012daa30b45ab94c5d086fa8942ec', 'metadata': {'bar': 'foo', 'foo': 'bar'}}

Custom OpenAI API Endpoint

Zep uses OpenAI LLM models for summarization and intent analysis tasks.

The OpenAI API Endpoint URL can now be customized, allowing Zep to connect to alternative APIs provided by both OpenAI and other software vendors and products.

This can be configured in the Zep config.yaml :

llm:
  # gpt-3.5-turbo or gpt-4
  model: "gpt-3.5-turbo"
  # Use only with an alternate OpenAI-compatible API endpoint
  openai_endpoint:
  openai_org_id:

Thanks to Drew Baumann for contributing this!

We also recently announced support for Azure OpenAI Endpoints.

Kubernetes Deployment

We've added a Kubernetes template that you can use as a blueprint for your own deployments. See the Kubernetes Deployment instructions for more information.

Next Steps

• Zep Quick Start
• Other Deployment Options