Vapi

Give a Vapi phone assistant a memory of each caller. Vapi calls your server at three moments: when a call starts, when the assistant uses a tool, and when the call ends. Answer the first with what matters about the caller, the second with a lookup, and save the call at the third. The next call starts knowing what was said in this one.

Install

Terminal

pip install fastapi uvicorn geniffy

Terminal

uv add fastapi uvicorn geniffy

Set GENIFFY_API_KEY from API keys in the Geniffy app, and VAPI_SECRET to a secret you also give Vapi.

Your server

server.py

import hmac
import json
import os

from fastapi import FastAPI, Header, HTTPException, Request
from geniffy import AsyncGeniffy

geniffy = AsyncGeniffy()                          # reads GENIFFY_API_KEY
app = FastAPI()
ASSISTANT_ID = os.environ.get("VAPI_ASSISTANT_ID", "your-assistant-id")


def space_of(message: dict) -> str:
    """The caller, by the digits of their number. If you know callers as customers, use your customer id, and
    the same memory serves their calls, chats and emails."""
    number = ((message.get("call") or {}).get("customer") or {}).get("number") or ""
    digits = "".join(ch for ch in number if ch.isdigit())
    if not digits:                                # never fall back to one shared memory for unknown callers
        raise HTTPException(400, "This call has no caller number.")
    return f"user_{digits}"


@app.post("/vapi")
async def vapi(request: Request, x_vapi_secret: str = Header("")):
    if not hmac.compare_digest(x_vapi_secret, os.environ["VAPI_SECRET"]):
        raise HTTPException(401)
    message = (await request.json())["message"]
    mem = geniffy.space(space_of(message))

    if message["type"] == "assistant-request":    # a call is starting: what to know before talking to them
        brief = await mem.brief(limit=12)
        known = "\n".join(f"- {m['text']}" for m in brief["memories"]) or (
            "Nothing is known about this caller yet. If they mention an earlier call, say so rather than guessing.")
        return {"assistantId": ASSISTANT_ID, "assistantOverrides": {"variableValues": {"memory": known}}}

    if message["type"] == "tool-calls":           # the assistant looks something up mid-call
        results = []
        for call in message.get("toolCallList") or []:
            args = call.get("parameters") or call.get("arguments") or {}
            args = json.loads(args) if isinstance(args, str) else args
            results.append({"name": call.get("name"), "toolCallId": call["id"],
                            "result": await mem.context(args.get("query", ""))})
        return {"results": results}

    if message["type"] == "end-of-call-report":   # the call is over: save who said what
        turns = [{"role": "assistant" if m["role"] in ("assistant", "bot") else "user", "content": m["message"]}
                 for m in (message.get("artifact") or {}).get("messages") or []
                 if m.get("role") in ("user", "assistant", "bot") and m.get("message")]
        if turns:
            await mem.memories.add(messages=turns, title="Call")
    return {}

Run it with uvicorn server:app, somewhere Vapi can reach over HTTPS.

A call starts with no question yet, so the assistant is given brief(): the caller's most load-bearing memories, newest first. Once the caller asks something, recall answers with context(), which holds only what bears on the question, each line with where it came from.

In Vapi

In your assistant's system prompt, put the memory where you want it:

You are a friendly phone assistant. Keep answers short.

<memory>
{{memory}}
</memory>

Add a function tool named recall, with one string parameter, query, and the description Look up what is known about the caller, with where it came from. Point its server URL at https://your-server/vapi.
On your phone number, set the server URL to https://your-server/vapi, so each call asks it for the assistant, and turn on the end-of-call report.
Make a credential for the header X-Vapi-Secret with your VAPI_SECRET, and use it on both server URLs.

When nothing is known about a caller, the memory says so in one sentence, and so does recall, so the assistant says it doesn't know instead of guessing. Each call is added as one conversation titled Call. Geniffy keeps who said what, so what the caller said becomes a fact about them, and what your assistant said stays the assistant's.

Last updated October 6, 2026