Migration Guide: Dialogflow CX LlamaIndex RAG Agent (OpenAI Responses API)
## The Architectural Shift
Dialogflow CX is a **deterministic finite state machine** : Flows → Pages → Routes → Intents. You author every path. The 2025–2026 rebrand to "Conversational Agents" on Vertex AI acknowledges this is giving way to an LLM instruction-following model (Root Agent → Sub-Agents → Tools), but the classic flow engine still underpins the runtime.
LlamaIndex + OpenAI Responses API is the opposite: **retrieval-augmented generation with no pre-authored paths**. You feed it documents, and the model decides what to retrieve and how to respond. The `OpenAIResponses` class (not the legacy `OpenAI` class) is the current integration point — it uses the Responses API code path, not chat/completions.
## What Breaks
### 1. Intents become retrieval queries
In CX, you have 50–200 intents with training phrases. Each intent maps to a fulfillment. In the RAG agent, there are no intents. The user's message is embedded, matched against your document corpus, and the top-k chunks are stuffed into the prompt.
**What this means in practice:** Your "What's your refund policy?" intent with 12 training phrases becomes a paragraph in a knowledge-base document. If your document is poorly chunked or the embedding model doesn't capture the semantic distance between "refund" and "money back," the retrieval misses. You lose the explicit mapping.
### 2. Context parameters become memory management
CX passes `context-parameters` (e.g., `order_id`, `customer_email`) between pages deterministically. In the LlamaIndex agent, conversation state lives in a `ChatMemoryBuffer` (default token limit: 3000). There's no schema enforcement. If the model "forgets" the order ID after three turns, your downstream tool call fails silently.
**Fix:** Use structured state (Pydantic models) alongside the memory buffer, or switch to a `FunctionAgent` with explicit tool parameters that the model must fill.
### 3. Webhooks become tool calls
CX webhooks are HTTP POSTs to your backend with a defined request/response schema. In LlamaIndex, you define Python functions as tools:
from llama_index.core.tools import FunctionTool
def check_order_status(order_id: str) -> str:
"""Look up order status by ID."""
# your backend call
return f"Order {order_id}: Shipped"
order_tool = FunctionTool.from_defaults(fn=check_order_status)
The model decides _when_ to call it. You lose the deterministic trigger. A user saying "where's my stuff?" might or might not trigger the tool depending on what's in the retrieval context.
### 4. No-match handling is now a hallucination risk
CX has `sys.no-match-default` — a clean fallback. The RAG agent has no equivalent. If retrieval returns irrelevant chunks, the model will often **confidently answer from the wrong document** rather than say "I don't know." This is the single biggest behavioral regression.
### 5. The OpenAI SDK version cliff
As of 2026, `openai>=3.0,<4` is the required pairing with `llama-index>=0.14.24`. The Responses API is the default code path. If your existing code uses `openai.ChatCompletion.create()` or the v2 client, it won't work. The `OpenAIResponses` class handles the new API surface (built-in tools, structured prediction via `responses.parse`, streaming, async).
## What You Gain
Capability | CX | LlamaIndex RAG
---|---|---
Open-domain questions | "I don't understand" | Answers from your corpus
New knowledge | Re-author intent + training phrases | Drop a PDF, re-index
Multi-turn reasoning | Manual context parameter threading | Automatic (with memory buffer)
Tool use | Webhook per intent | Model decides when to call
Iteration speed | Console + deploy per change | Edit a document, re-query
Cost model | Per session (Google) | Per token (OpenAI) + infra
The biggest gain: **your knowledge base is now a data problem, not a conversation-design problem.** Adding a new policy means uploading a document, not authoring 8 intents with 40 training phrases.
## The Testing Gap
This is where the migration is genuinely hard. Here's what you lose and what you need to build:
### What CX gave you for free
* **Test cases per intent** (you could attach 5 test utterances per intent and get a pass/fail in the console)
* **Deterministic replay** (same input → same path, every time)
* **Structured failure modes** (no-match, no-input, invalid-parameter are distinct events you can alert on)
### What you must build
**1. Retrieval quality evals (the new "intent accuracy")**
Your test cases are now: _given query Q, does retrieval return the correct document chunk?_ Use Ragas or DeepEval:
from ragas.metrics import faithfulness, answer_relevancy, context_precision
from ragas import evaluate
from datasets import Dataset
eval_data = Dataset.from_dict({
"question": ["What's the refund window?", ...],
"answer": [agent_response, ...],
"contexts": [retrieved_chunks, ...],
"ground_truth": ["30 days from delivery", ...],
})
result = evaluate(dataset=eval_data, metrics=[faithfulness, context_precision])
**2. Hallucination detection (replaces no-match)**
You need an LLM-as-judge that scores: _is this answer grounded in the retrieved context, or is the model inventing?_ Gate deploys on faithfulness > 0.85.
**3. Tool-call regression (replaces webhook integration tests)**
For each tool, write a test that asserts: _given a query that should trigger the tool, does the agent actually call it with the right parameters?_ This is non-deterministic, so run N=10 and require ≥9/10.
**4. Multi-turn state tests**
CX context parameters were testable because they were explicit. Now you need conversation-level tests: _does the agent retain the order_id across 4 turns before calling the status tool?_ DeepEval's `KnowledgeRetention` and `ConversationCompleteness` metrics cover this.
**5. The gap no tool fills yet**
There's no equivalent of CX's "simulate 1000 sessions and report intent distribution." You need to build a **simulation flywheel** : generate synthetic multi-turn conversations (using a different model as the "user"), run them through your agent, and score the traces. DoorDash reportedly reduced hallucination ~90% this way.
## Concrete Skeleton
# agent.py
import os
from llama_index.core import VectorStoreIndex, Settings, SimpleDirectoryReader
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAIResponses
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core.memory import ChatMemoryBuffer
# --- Configuration ---
Settings.llm = OpenAIResponses(model="gpt-4o-mini", temperature=0.1)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
# --- Ingestion (replaces intent authoring) ---
documents = SimpleDirectoryReader("./knowledge_base", recursive=True).load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=4)
# --- Tools (replaces webhooks) ---
def lookup_order(order_id: str) -> str:
"""Look up order status by order ID."""
...
def check_eligibility(customer_email: str) -> str:
"""Check refund eligibility for a customer."""
...
from llama_index.core.tools import FunctionTool
tools = [
FunctionTool.from_defaults(fn=lookup_order),
FunctionTool.from_defaults(fn=check_eligibility),
FunctionTool.from_defaults(
fn=lambda q: str(query_engine.query(q)),
name="search_knowledge_base",
description="Search internal documentation for policy answers.",
),
]
# --- Agent (replaces the entire flow) ---
agent = FunctionAgent(
tools=tools,
llm=Settings.llm,
system_prompt=(
"You are a customer support agent. Use the knowledge base for policy "
"questions. Use tools for account-specific lookups. If you cannot "
"find an answer in the knowledge base, say so explicitly — do not "
"guess."
),
)
The system prompt line "do not guess" is your no-match handler. It's weaker than CX's `sys.no-match-default`, but it's the starting point.
## Migration Sequence
1. **Export your CX knowledge** — pull all fulfillment text, intent descriptions, and webhook response schemas into a document corpus.
2. **Build the index** — chunk with `SentenceSplitter(chunk_size=512, chunk_overlap=50)`. Validate retrieval on 50 known query→answer pairs before wiring the agent.
3. **Port webhooks to tools** — one `FunctionTool` per webhook. Keep the same backend endpoints.
4. **Replace the flow with a single agent** — start with one `FunctionAgent`. Don't build a multi-agent hierarchy until the single agent passes evals.
5. **Build the eval suite before you cut traffic** — 200 test cases minimum. Gate on faithfulness, context precision, and tool-call accuracy.
6. **Run in shadow mode** — route 10% of live traffic to the RAG agent, log responses, compare against CX. Iterate for 2 weeks.
7. **Cut over** — flip the traffic split. Keep CX running for 30 days as a rollback.
The version pin: `llama-index>=0.14.24`, `openai>=3.0,<4`. Both default to the Responses API. Don't use the legacy `OpenAI` class — it's the chat/completions path and will drift.