Pipecat
Give a Pipecat voice agent a memory of each caller. One frame processor sits in front of the LLM: before each reply it puts the caller's briefing in front of the LLM, cued by what they just said, and it saves the call as it goes. The next call starts knowing what was said in this one.
Install
pip install "geniffy[pipecat]"uv add "geniffy[pipecat]"It needs Python 3.11 or later and Pipecat 1.9 or later. Add the speech, LLM and transport services your bot
already uses. Set GENIFFY_API_KEY from API keys in the Geniffy app.
In your pipeline
Put GeniffyMemory between the user aggregator and the LLM, with one memory per caller. Your system prompt stays
on your LLM service, as system_instruction in its settings, such as
OpenAILLMService(settings=OpenAILLMService.Settings(system_instruction=...)):
from geniffy import AsyncGeniffy
from geniffy.pipecat import GeniffyMemory
from pipecat.pipeline.pipeline import Pipeline
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair
geniffy = AsyncGeniffy() # once for the process; reads GENIFFY_API_KEY
def build_pipeline(transport, stt, llm, tts, caller_id: str) -> Pipeline:
memory = GeniffyMemory(geniffy.space(f"user_{caller_id}")) # your own id for the caller
aggregators = LLMContextAggregatorPair(LLMContext())
return Pipeline([
transport.input(),
stt,
aggregators.user(),
memory, # the caller's briefing, before the LLM
llm,
tts,
transport.output(),
aggregators.assistant(),
])Before each reply the processor reads the briefing with what the caller just said as the cue: where things stand with them, what is due or was promised, what happened on earlier calls, then the facts, each dated. The briefing goes in a message of its own just before their words, replaced each turn, and every other message stays as it was. It is read, not written by a model, so it comes back in a fraction of a second. The call's first briefing is read while the call connects, and the first reply waits up to 1.5 seconds for it; a later reply waits up to 0.3 seconds for one cued by the new words, and past that goes on with the last one. When nothing is known, the briefing says so in one sentence, so the agent says it doesn't know instead of guessing; when it could not be read, the agent is told the caller's history is not at hand, never that there is none.
When something changes between calls, the next briefing has the new value: "I'll pay on Friday", then "make it Monday" on a later call, and the briefing says Monday, with Friday kept in the memory as history.
The call, saved as it goes
The call is saved into one conversation titled Call as it goes: after each reply, what was said since the last
save, and the rest once the pipeline has shut down, whether the call ended or was cancelled. A call of any length is
kept, and a bot that stops keeps what came before. Geniffy keeps who said what, so what the caller said becomes a
fact about them, and what your agent said stays the agent's.
Options
| Option | Default | What it does |
|---|---|---|
instructions |
"" |
Text to put before the briefing, in its message; your system prompt belongs on the LLM service |
project |
none | Brief on one project only, as your sources are labelled |
budget_chars |
2500 |
How much briefing a reply carries; voice prompts stay short |
timeout |
1.5 |
The longest the call's first reply waits for the briefing, in seconds |
turn_timeout |
0.3 |
The longest a later reply waits for a briefing cued by the caller's new words, in seconds |
save |
True |
Save the call as it goes |
title |
"Call" |
The title the saved call carries |
In production
- Pipecat logs what it sends the LLM at
DEBUG, Loguru's default level, so each caller's briefing goes into your logs on every turn. Log atINFOin production:logger.remove(), thenlogger.add(sys.stderr, level="INFO"). - Key the memory on your own id for the caller. If all you have is their number, it says who is probably calling,
not who is, and a withheld one is no number at all: leave
GeniffyMemoryout of the pipeline for a withheld number, so nothing is read or saved, and see the production checklist before your agent shares anything private. - A caller who asks to be forgotten is one call:
await geniffy.forget_space(f"user_{caller_id}").