AI agent integration
Wire ConvoMem into any LLM agent as callable tools — with ready-to-paste tool definitions, SDK references, and channel blueprints.
ConvoMem is designed to work as a set of tools an LLM agent can call. Register the definitions below with your platform (OpenAI, Anthropic, VAPI, Retell, Bland.ai, etc.) and ConvoMem handles identity resolution, session management, and memory extraction.
The agent loop
Every integration follows the same pattern:
1. recall_customer_context → at conversation start: inject the brief into the system prompt
2. LLM generates reply → send to user
3. capture_message → after every exchange (fire-and-forget)
search_memories → when a turn needs a specific past fact
repeat for each turn
4. end_conversation → final turns + close, on goodbye / issue resolved
escalate_conversation → final turns + escalate, on transfer to a human
(call get_handoff_brief first)
Don't let the model decide which turns are "important" before capturing. Send every exchange verbatim and let ConvoMem's pipeline extract and de-duplicate. Descriptions like "call after each meaningful exchange" cause models to drop turns — you lose memories. The tool descriptions below are worded to prevent this.
capture_message returns immediately. Extraction runs in the background. Don't
block the user's reply waiting for it — call it without await after the response
is sent.
Tool definitions
The JSON below is OpenAI function-calling format. Map name, description, and
parameters into your platform's tool schema — the fields are the same everywhere.
The description is the only instruction the model gets per tool, so wording matters.
Tighten the descriptions to your domain, but keep the guarantees — especially the
verbatim capture rule.
recall_customer_context
Call once at conversation start, as soon as you have an identifier. This is the cold open: it returns the customer profile and a call-opening brief that leads with any unresolved issue, then the customer's most trusted memories. It does no vector search, so it is fast enough to run while a call is still connecting.
{
"type": "function",
"function": {
"name": "recall_customer_context",
"description": "Retrieve what is already known about the customer before you reply. CALL IT once at the start of the conversation, as soon as you have any identifier (phone, email, externalId, or customerId). USE the returned `context` string as background when forming your reply, mention any `openIssue` proactively, and keep the returned `customer.id` for later tool calls. Pass `autoCreate: true` so a first-time caller is still registered. For a specific question later in the conversation, use search_memories instead. This is a read — it never stores conversation turns.",
"parameters": {
"type": "object",
"properties": {
"phone": { "type": "string", "description": "Caller phone number, ideally E.164 (+14155550101)." },
"email": { "type": "string", "description": "Customer email address." },
"externalId": { "type": "string", "description": "Your CRM / system customer ID." },
"customerId": { "type": "string", "description": "ConvoMem customer UUID, if already known." },
"autoCreate": { "type": "boolean", "description": "Create the customer if not found.", "default": true },
"userName": { "type": "string", "description": "Customer's display name, to set or backfill the profile when they introduce themselves." }
}
}
}
}const { customer, context, openIssue } = await client.coldOpen(
{ email, phone, customerId },
{ autoCreate: true },
)The response contains found, isNewCustomer, customer (the profile, or null when
the customer does not exist and autoCreate is off), context, tokenCount,
memories, and openIssue when an issue is still open.
capture_message
Call after every exchange — the user turn and your reply, in order.
{
"type": "function",
"function": {
"name": "capture_message",
"description": "Record the conversation into ConvoMem's long-term memory. CALL IT after EVERY exchange — i.e. after you send each reply — passing the new turns since your last capture (the user's message AND your assistant reply, in order). Do NOT judge whether a turn is 'important' or 'memorable' and do NOT skip, summarize, paraphrase, or filter turns: send them verbatim and let ConvoMem decide what to extract and de-duplicate. Skipping turns loses memories. Each call appends to the customer's active session automatically, so send only the new turns, not the whole transcript again. For the last exchange of the conversation, use end_conversation or escalate_conversation instead. This is fire-and-forget — extraction happens in the background, so it returns before facts are searchable.",
"parameters": {
"type": "object",
"properties": {
"customerId": { "type": "string", "description": "Customer UUID from recall_customer_context. Preferred identifier." },
"phoneNumber": { "type": "string", "description": "Phone identity (use if customerId is unknown)." },
"email": { "type": "string", "description": "Email identity (use if customerId is unknown)." },
"externalId": { "type": "string", "description": "External / CRM identity (use if customerId is unknown)." },
"messages": { "type": "string", "description": "JSON-encoded array of the NEW turns since your last capture, in chronological order, verbatim: [{\"role\":\"user\",\"content\":\"…\"},{\"role\":\"assistant\",\"content\":\"…\"}]. Include both the user turn and your reply. Never omit a turn." },
"channel": { "type": "string", "enum": ["VOICE", "CHAT", "SMS", "EMAIL"], "description": "The channel this conversation is on. Use the same value for the whole conversation.", "default": "CHAT" },
"userName": { "type": "string", "description": "The customer's name if they state it during the conversation." },
"idempotencyKey": { "type": "string", "description": "Optional unique key (≤128 chars) per exchange to make retries safe without double-recording." }
},
"required": ["messages"]
}
}
}// fire-and-forget — don't await
client.capture(
[{ role: 'user', content: userMsg }, { role: 'assistant', content: reply }],
{ customerId },
{ channel: 'CHAT' },
).catch(console.error)Send only new turns each call — not the full transcript. Across the conversation, every turn is captured exactly once.
search_memories
Targeted mid-conversation recall for a specific topic.
{
"type": "function",
"function": {
"name": "search_memories",
"description": "Search a customer's long-term memories for a specific topic and get a prompt-ready `context` string. Use this — instead of recall_customer_context — when the user asks about or refers to something specific from the past (e.g. 'what plan am I on?', 'did I report this before?'). Set `topic` to the exact thing you're trying to recall. Read-only — stores nothing.",
"parameters": {
"type": "object",
"properties": {
"topic": { "type": "string", "description": "Topic or question to find relevant memories for." },
"customerId": { "type": "string", "description": "Customer UUID from recall_customer_context." },
"phone": { "type": "string", "description": "Customer phone number." },
"email": { "type": "string", "description": "Customer email." },
"externalId": { "type": "string", "description": "External / CRM identifier." }
},
"required": ["topic"]
}
}
}const { context } = await client.lookup(topic, { email, phone, externalId })get_handoff_brief
Call immediately before transferring to a human agent.
{
"type": "function",
"function": {
"name": "get_handoff_brief",
"description": "Build a cross-channel briefing (full journey across chat/email/voice/SMS, key memories, commitments already made, sentiment trend, open-issue detection, and a narrative) for the human who is about to take over. CALL IT immediately before you transfer or escalate, so the agent has full context. Relay the `narrative` and `openIssue.summary` to the human agent; do not read raw memory IDs to the customer. Read-only.",
"parameters": {
"type": "object",
"properties": {
"phone": { "type": "string" },
"email": { "type": "string" },
"externalId": { "type": "string" },
"customerId": { "type": "string" },
"narrative": { "type": "string", "enum": ["true", "false"], "description": "Set 'false' to skip the narrative for a faster, structured-only brief.", "default": "true" }
}
}
}
}const handoff = await client.getHandoff({ customerId, email, phone })end_conversation
Call once, when the conversation is genuinely finished. It sends the final turns and
the close in one capture request (endConversation: true), so ConvoMem persists the
transcript before it closes the conversation and runs analysis. Closing in a separate
request right after a capture can race that ordering.
{
"type": "function",
"function": {
"name": "end_conversation",
"description": "Record the final turns of the conversation AND close it as completed, in one call. CALL IT once, instead of capture_message, for the last exchange — when the user says goodbye, confirms their issue is resolved, or the call ends. Do NOT call it mid-conversation or because there is a pause. Pass the new turns since your last capture, verbatim, exactly as for capture_message. Set `outcome` to a one-line factual result (what was done or decided), not a transcript. After this, the next message from the customer starts a fresh conversation.",
"parameters": {
"type": "object",
"properties": {
"customerId": { "type": "string", "description": "Customer UUID from recall_customer_context. Preferred identifier." },
"phoneNumber": { "type": "string", "description": "Phone identity (use if customerId is unknown)." },
"email": { "type": "string", "description": "Email identity (use if customerId is unknown)." },
"externalId": { "type": "string", "description": "External / CRM identity (use if customerId is unknown)." },
"messages": { "type": "string", "description": "JSON-encoded array of the final turns since your last capture, in chronological order, verbatim." },
"channel": { "type": "string", "enum": ["VOICE", "CHAT", "SMS", "EMAIL"], "default": "CHAT" },
"outcome": { "type": "string", "description": "Short resolution summary (≤500 chars), e.g. 'Booked return flight; customer satisfied.'" }
},
"required": ["messages"]
}
}
}await client.capture(finalTurns, { customerId }, {
channel: 'CHAT',
endConversation: true,
outcome: 'Resolved billing issue',
})escalate_conversation
Call when handing off to a human. Like end_conversation, it sends the final turns
and the state change together (escalate: true). Call get_handoff_brief first so the
human has full context.
{
"type": "function",
"function": {
"name": "escalate_conversation",
"description": "Record the final turns AND hand the conversation to a human agent, marking it ESCALATED, in one call. CALL IT, instead of capture_message, when the user explicitly asks for a person, is clearly frustrated or upset, or the request is outside what you can resolve. Use this rather than end_conversation when a human still needs to act. Pass the new turns since your last capture, verbatim. Set `reason` to a short factual explanation of why (≤500 chars). Call get_handoff_brief first so the human has full context.",
"parameters": {
"type": "object",
"properties": {
"customerId": { "type": "string", "description": "Customer UUID from recall_customer_context. Preferred identifier." },
"phoneNumber": { "type": "string", "description": "Phone identity (use if customerId is unknown)." },
"email": { "type": "string", "description": "Email identity (use if customerId is unknown)." },
"externalId": { "type": "string", "description": "External / CRM identity (use if customerId is unknown)." },
"messages": { "type": "string", "description": "JSON-encoded array of the final turns since your last capture, in chronological order, verbatim." },
"channel": { "type": "string", "enum": ["VOICE", "CHAT", "SMS", "EMAIL"], "default": "CHAT" },
"reason": { "type": "string", "description": "Why it is being escalated (≤500 chars)." }
},
"required": ["messages"]
}
}
}await client.capture(finalTurns, { customerId }, {
channel: 'CHAT',
escalate: true,
reason: 'Complex billing dispute',
})endConversation and escalate are mutually exclusive; sending both returns 422.
To close or escalate a conversation when there are no turns left to send, use the
Conversations endpoints. The flat routes
(POST /customers/conversations/end and /escalate) resolve the customer from any
identity field and act on that customer's active conversation; they do not take a
conversationId, and return 404 when no conversation is active.
URL & SDK reference
| Tool | API endpoint | TypeScript | Python | Rust |
|---|---|---|---|---|
recall_customer_context | GET /customers/lookup | client.coldOpen(identity, { autoCreate: true }) | client.cold_open(identity, auto_create=True) | client.cold_open(&id, Some(true), None, None) |
capture_message | POST /capture | client.capture(msgs, identity, { channel }) | client.capture(msgs, identity, channel=…) | client.capture(msgs, &id, channel, None, None) |
search_memories | GET /customers/memories/lookup | client.lookup(topic, identity) | client.lookup(topic, identity) | client.lookup(topic, &id, None, None, None) |
get_handoff_brief | GET /customers/handoff | client.getHandoff(identity) | client.get_handoff(identity) | client.get_handoff(&id, None, None) |
end_conversation | POST /capture with endConversation: true | client.capture(msgs, identity, { endConversation: true, outcome }) | client.capture(msgs, identity, end_conversation=True, outcome=…) | client.capture_with(&CaptureRequest { end_conversation: Some(true), … }) |
escalate_conversation | POST /capture with escalate: true | client.capture(msgs, identity, { escalate: true, reason }) | client.capture(msgs, identity, escalate=True, reason=…) | client.capture_with(&CaptureRequest { escalate: Some(true), … }) |
All endpoints under https://api.convomem.com/api/v1. Auth header: X-API-Key: <key>.
SDK docs: TypeScript · Python · Rust
Channel blueprints
Chatbot
Web widget, WhatsApp, Slack — tight recall → reply → capture per message.
import OpenAI from 'openai'
import { ConvoMem } from '@optraze/convomem-sdk'
const client = new ConvoMem({ apiKey: process.env.CONVOMEM_API_KEY! })
const openai = new OpenAI()
async function handleMessage(userMessage: string, session: { email: string }) {
// 1. Recall — before every reply
const { context } = await client.lookup(userMessage, {
email: session.email,
})
// 2. Reply with context injected
const completion = await openai.chat.completions.create({
model: 'gpt-4o',
messages: [
{
role: 'system',
content: `You are a helpful support agent.\n\nWhat you know about this customer:\n${context}`,
},
{ role: 'user', content: userMessage },
],
})
const reply = completion.choices[0].message.content!
// 3. Capture — fire-and-forget
client.capture(
[
{ role: 'user', content: userMessage },
{ role: 'assistant', content: reply },
],
{ email: session.email },
{ channel: 'CHAT' },
).catch(console.error)
return reply
}Voicebot / IVR
Sub-second latency budgets — run the cold open at call start, register tools for inline calls during.
Call start:
// Cold open as soon as the call connects — before the first reply
const { context } = await client.coldOpen(
{ phone: callerPhoneNumber },
{ autoCreate: true },
)
const systemPrompt = `You are a voice support agent.
What you know about this caller:
${context || 'No prior history.'}`Call end:
// Send the remaining turns and close the conversation in one ordered request
await client.capture(remainingTurns, { phone: callerPhoneNumber }, {
channel: 'VOICE',
endConversation: true,
outcome: 'Resolved billing question',
})VAPI — add an API Tool: method GET, URL
https://api.convomem.com/api/v1/customers/lookup, header X-API-Key: sk-org-…, map
caller number to the phone query param.
Bland.ai — register a custom tool:
{
"name": "lookup_customer",
"description": "Fetch caller history and memory context.",
"url": "https://api.convomem.com/api/v1/customers/lookup?phone={{phone}}&autoCreate=true",
"method": "GET",
"headers": { "X-API-Key": "sk-org-…" }
}Email bot
Turn-based with long windows — use the sender address as identity.
async function handleInboundEmail(from: string, subject: string, body: string) {
// 1. Recall context for this sender
const { context } = await client.lookup(`${subject} ${body}`, { email: from })
// 2. Draft and send reply
const reply = await generateReply(context, subject, body)
// 3. Capture the exchange — and close the conversation if the thread is resolved
const resolved = isResolved(reply)
await client.capture(
[
{ role: 'user', content: `Subject: ${subject}\n\n${body}` },
{ role: 'assistant', content: reply },
],
{ email: from },
{
channel: 'EMAIL',
...(resolved ? { endConversation: true, outcome: 'Email thread resolved' } : {}),
},
)
return reply
}Conversation sessions
Conversations idle-close after 15 minutes of inactivity (configurable server-side). You don't need to close conversations explicitly unless you want to:
- Record an
outcomefor handoff briefings - Trigger the
conversation.completedwebhook immediately - Force a new session to start on the next message
The next capture after a closed session automatically starts a fresh conversation.
What's next
- Capture — full capture API reference and idempotency
- Handoff — briefing response fields and injection patterns
- Webhooks — react to
memory.capturedandconversation.*events - Errors & limits — rate limits, error codes, retry guidance
SDK: TypeScript · Python · Rust