Retell

Give a Retell voice agent a memory of each caller. Retell calls your server when a call comes in, when the agent uses a custom function, and when the call ends. Answer the first with what matters about the caller, the second with a lookup, and save the call at the third. The next call starts knowing what was said in this one.

Install

Terminal

pip install fastapi uvicorn retell-sdk geniffy

Terminal

uv add fastapi uvicorn retell-sdk geniffy

Set GENIFFY_API_KEY from API keys in the Geniffy app, and RETELL_API_KEY to your Retell key, which signs every request Retell sends you.

Your server

server.py

import json
import os

from fastapi import FastAPI, HTTPException, Request
from geniffy import AsyncGeniffy
from retell import Retell

geniffy = AsyncGeniffy()                          # reads GENIFFY_API_KEY
retell = Retell(api_key=os.environ["RETELL_API_KEY"])
app = FastAPI()


def space_for(number: str | None) -> str:
    """The caller, by the digits of their number. If you know callers as customers, use your customer id, and
    the same memory serves their calls, chats and emails."""
    digits = "".join(ch for ch in number or "" if ch.isdigit())
    if not digits:                                # never fall back to one shared memory for unknown callers
        raise HTTPException(400, "This call has no caller number.")
    return f"user_{digits}"


async def signed(request: Request) -> dict:
    """The request, once its X-Retell-Signature shows Retell sent it."""
    body = (await request.body()).decode()
    if not retell.verify(body, api_key=os.environ["RETELL_API_KEY"],
                         signature=request.headers.get("X-Retell-Signature", "")):
        raise HTTPException(401)
    return json.loads(body)


@app.post("/retell/webhook")
async def webhook(request: Request):
    event = await signed(request)
    if event.get("event") == "call_inbound":      # a call is coming in: what to know before answering
        mem = geniffy.space(space_for(event["call_inbound"].get("from_number")))
        brief = await mem.brief(limit=12)
        known = "\n".join(f"- {m['text']}" for m in brief["memories"]) or (
            "Nothing is known about this caller yet. If they mention an earlier call, say so rather than guessing.")
        return {"call_inbound": {"dynamic_variables": {"memory": known}}}
    if event.get("event") == "call_ended":        # the call is over: save who said what
        call = event.get("call") or {}
        turns = [{"role": "assistant" if t["role"] == "agent" else "user", "content": t["content"]}
                 for t in call.get("transcript_object") or []
                 if t.get("role") in ("agent", "user") and t.get("content")]
        if turns:
            await geniffy.space(space_for(call.get("from_number"))).memories.add(messages=turns, title="Call")
    return {}


@app.post("/retell/recall")
async def recall(request: Request):                # the custom function the agent calls mid-call
    payload = await signed(request)
    mem = geniffy.space(space_for((payload.get("call") or {}).get("from_number")))
    return {"memory": await mem.context((payload.get("args") or {}).get("query", ""))}

Run it with uvicorn server:app, somewhere Retell can reach over HTTPS.

A call comes in with no question yet, so the agent is given brief(): the caller's most load-bearing memories, newest first. Once the caller asks something, the recall function answers with context(), which holds only what bears on the question, each line with where it came from.

In Retell

In your agent's prompt, put the memory where you want it:

You are a friendly phone assistant. Keep answers short.

<memory>
{{memory}}
</memory>

On your phone number, set the inbound webhook to https://your-server/retell/webhook.
On your agent, set the webhook URL to the same address, so it hears when a call ends.
Add a custom function named recall, with one string parameter, query, and the description Look up what is known about the caller, with where it came from. Point it at https://your-server/retell/recall.

When nothing is known about a caller, the memory says so in one sentence, and so does recall, so the agent says it doesn't know instead of guessing. Each call is added as one conversation titled Call. Geniffy keeps who said what, so what the caller said becomes a fact about them, and what your agent said stays the agent's.

Last updated October 6, 2026