If you’re evaluating conversational AI for support, sales, or internal tooling, you’ve probably noticed that vendors use “chatbot” and “AI agent” to mean very different things depending on what they’re selling. Some products labeled “AI agent” are thin wrappers around a language model with no tool access. Some products labeled “chatbot” do more autonomous reasoning than systems calling themselves agents.
A chatbot is a conversational interface; an AI agent is one possible way to control the work behind it. A scripted chatbot can call APIs and retain state. An agent can choose its next step from permitted tools, but that does not establish the user’s identity, authorize a write, or prove a task succeeded. This post compares three working architectures and uses one hypothetical return request to make those boundaries concrete.
Optional example checks
-
Compare deployment methods. Check your declared channels, website access and required features against Quickchat AI setup options. This does not inspect your website or account.
-
Check plan requirements. Compare declared usage, content, team and feature needs with public Quickchat AI plan bundles. This is not a price quote or account check.
A note on terminology
The industry hasn’t settled on consistent definitions, so this post uses three working categories rather than treating product labels as specifications:
-
Rule-based chatbot: Intent classification, entity extraction and scripted dialogue flows choose the next step. Responses use templates rather than an LLM. The workflow can retain state, call APIs and enforce approvals.
-
Answer-only LLM chatbot: A language model answers from supplied context, often retrieved from a knowledge base. In this article’s example it has no action tools. Other products called chatbots or copilots can have tools, multiple model calls and human approvals.
-
AI agent: An LLM selects steps and tools within a configured loop, using observations to decide what to do next. Memory, permissions, stopping conditions and human review depend on the implementation. Autonomy is bounded by the tools and authority the system actually provides.
These are schematic examples. More model-directed orchestration does not guarantee better accuracy, lower cost or a completed business outcome.
A useful architectural distinction is who selects the next step: predefined application logic or the model. Anthropic’s guide to workflows and agents makes this distinction and recommends starting with the simplest solution that meets the task. Hybrid systems can combine both.
Consider a customer saying, “I received the wrong item. What is your return policy, and can you start a return?” The following is a hypothetical design example, not a recorded Quickchat AI run or a built-in refund workflow.
| Stage | Permitted work | Evidence needed before claiming success |
|---|---|---|
| Answer the policy question | Retrieve the published policy without exposing private order data | The applicable policy source; an answer does not establish this order’s eligibility |
| Read the order | Use an authenticated customer context and an API that enforces access to the matching order | An authorized response with the necessary order fields; an email or order number typed in chat is not authentication |
| Request a return | Separately check write permission, policy conditions and any required customer confirmation | A confirmed result from the return system, such as an accepted request identifier; this is not a paid refund or completed return |
| Hand off an exception | Stop for missing authority, unsupported policy exceptions or an uncertain write result | A recorded handoff request and its known destination or queue; no promise that a person has replied |
A deterministic workflow can implement all four stages. An answer-only system can explain the policy and direct the customer to an authenticated service. An agent can choose among configured steps, but it must satisfy the same access and evidence requirements.
What is a rule-based chatbot?
A rule-based chatbot conducts conversations with users through text or voice using predefined logic. It is built on a combination of intent classification, entity extraction, and scripted dialogue flows.
How rule-based chatbots work
The typical chatbot architecture has three components:
-
Natural Language Understanding (NLU): Parses the user’s message to identify an intent (what the user wants) and extract entities (specific data like dates, product names, or order numbers). Most NLU systems use a combination of regex patterns, keyword matching, and trained classifiers.
-
Dialogue Manager: A state machine that determines the next step based on the current conversation state and the recognized intent. Each state has a set of possible transitions, and the dialogue manager follows the defined flow.
-
Response Generator: Retrieves a templated response associated with the current state. In simple chatbots, this is a lookup table. In slightly more sophisticated ones, the template includes slots that get filled with extracted entities.
A simplified hypothetical flow, after the service authenticates the customer and checks access to the order, looks like this:
User: "What's the status of order #12345?"
→ NLU: intent=order_status, entity={order_id: 12345}
→ Dialogue Manager: state=check_order → authorized order_status_api(12345)
→ Response: "Order #12345 shipped on April 7 and is expected by April 11."The architecture looks like this:
This works well for closed-domain problems where the set of possible intents is small and well-defined. An FAQ bot, an appointment scheduler, or an order-status checker can all be built this way.
Limitations of the rule-based architecture
A state machine needs explicit handling for the paths it supports. Its practical limits depend on the classifier, workflow design and integration coverage:
-
Intent coverage: Requests need to map to a supported intent or fallback. Unfamiliar wording can be misclassified, even though a trained classifier can generalize beyond exact phrases. Overlapping intents require evaluation and maintenance.
-
Predefined branching: The dialogue manager can combine stored information across turns and branch on it. New kinds of decisions still need rules or another decision component; the graph does not invent a new workflow at runtime.
-
Brittle entity extraction: Traditional slot-filling works for structured inputs (dates, numbers, product SKUs) but struggles with open-ended descriptions, ambiguous references, or multi-part requests.
-
Workflow maintenance: New use cases can require additional intents, branches, templates or integrations. Reusable components help, but maintenance effort must be measured rather than assumed to grow at a fixed rate.
For a deeper look at how chatbot architectures have evolved over the years, see From NLP Chatbots to Generative AI.
What is an AI agent?
Here, an AI agent is a system in which an LLM selects the next step or tool based on the task and observations. It can operate inside a broader scripted workflow. Persistent memory and permission to change external systems are optional capabilities, not consequences of the label.
The term “agent” also has a broader history in AI research: a system perceives its environment and acts toward a goal. This article focuses on LLM-based orchestration, where observation, planning, tool execution and evaluation can repeat until a stopping condition is reached.
How AI agents work
The core architecture of an AI agent is a reasoning loop (sometimes called a ReAct loop, after the ReAct paper that formalized the pattern). The loop has four phases:
-
Observe: The agent receives the user’s message along with conversation history, system instructions, and any retrieved context (e.g., relevant knowledge base articles).
-
Reason: The LLM uses the available context to select a response, clarification or tool call. Unlike a fixed transition in a state graph, that selection is made at runtime. It can be wrong and needs evaluation.
-
Act: The model proposes a structured call, such as
search_knowledge_base(query="return policy")orcreate_return_request(order_id=12345). Application controls and the destination API must enforce the permitted operation and scope before execution. A proposed call is not authorization. -
Evaluate: The system examines the result and decides whether to continue, respond or stop for human review. A model’s conclusion that it is finished is not independent evidence of a successful business operation. Bound the loop and verify external state where necessary.
The same return scenario can be expressed as a proposed control flow. No calls or outcomes below were executed or observed. The operation names are illustrative, not a Quickchat AI configuration you can paste into an account.
Customer: "I received the wrong item. Can you start a return?"
[Policy] Retrieve the applicable return policy and cite its conditions.
[Read] Require authenticated customer context and order-level access.
Look up the matching order through the authorized service.
Treat "wrong item received" as the customer's report unless verified.
[Gate] Check return policy, write authority and required confirmation.
If a condition is missing, ask for clarification or hand off.
[Write] Submit a return request only after the separate write gate passes.
[Check] Report acceptance only if the external result confirms it.
An accepted request does not mean a refund was paid.
If the result is uncertain, inspect state or hand off; do not retry blindly.
[Handoff] Record unresolved facts and the known queue or recipient.
Do not claim that a human has already replied.The agentic part is selecting the next permitted step from context. The service still needs defined tools, policy rules, identity checks and failure handling. A fixed workflow could perform the same operations; the architecture changes orchestration, not the customer’s authority.
Tool use and function calling
Tool use (also called function calling) connects a model to retrieval or external operations. The LLM receives tool schemas with parameter names, types and descriptions. It can propose a call from the available set; the application decides whether that call is allowed and how its result is exposed.
Common tool categories for customer-facing AI agents include:
| Category | Examples |
|---|---|
| Knowledge retrieval | Search knowledge base, look up FAQ, retrieve product documentation |
| Data lookup | Check order status, fetch account details, query inventory |
| Write operations | Create a return request or ticket, update an allowed customer field; each requires its own authority |
| Communication | Send email, trigger SMS, post to Slack |
| Escalation | Request human review with relevant context and a known queue or recipient |
A schema describes the call shape; it does not authenticate a customer or enforce business policy. Validate parameters and access at the destination, restrict exposed fields, and distinguish a read from a state-changing operation. Prompt instructions alone are not a security boundary.
The tutorial on building an AI agent that takes real actions explains a described HTTP request called during a conversation. The follow-up on ordering Actions with run conditions and saved results shows Run only when and Save to memory. Those controls do not establish customer identity or make a write idempotent: trusted metadata and destination-side authorization remain necessary. These links preserve their own demo context; the hypothetical return flow here has not been tested.
Memory and context management
A scripted chatbot can store filled slots such as order_id=12345 and intent=order_status, alongside conversation history or other state. Agent implementations often combine several kinds of context to handle open-ended requests. Neither architecture automatically provides safe cross-session memory.
Useful context layers include:
- Conversation history: The raw sequence of messages in the current session.
- Working memory: Intermediate results from tool calls and reasoning steps that inform subsequent decisions.
- Retrieved context: Knowledge base articles, product documentation, or customer records pulled in via retrieval-augmented generation (RAG) based on the current query.
- Long-term memory (optional): Persistent facts about the user or account that carry across sessions (e.g., preferred language, past issues, subscription tier).
Managing context within the LLM’s token window is an engineering constraint. Summarization, sliding windows and selective retrieval can reduce context size, but may omit important details. Keep identity, permissions and authoritative transaction state in trusted systems rather than relying on a model summary.
The middle ground: LLM chatbots and copilots
An answer-only LLM chatbot can retrieve relevant knowledge, pass it with the user’s message to a model and return a response. This simple design has no model-directed action loop. It is one implementation pattern, not a definition of every product called a chatbot or copilot.
For policy questions, a retrieval-and-answer system may be sufficient. It can handle varied wording without requiring a separate intent for every phrasing. Retrieval quality, supported languages and source quality still need evaluation.
In the return example, an answer-only system explains the published policy but does not retrieve private order data or create a return request. Add an authenticated workflow or appropriately restricted tools if those operations are required. A scripted integration can also perform them.
Hybrid designs are common possibilities: an LLM classifies a request, a deterministic workflow enforces the required checks, and a model drafts the reply. Evaluate where decisions happen, which operations are allowed and what evidence is returned, rather than inferring those properties from “chatbot,” “copilot” or “agent.”
Side-by-side comparison
The middle column describes the answer-only example used in this article. Features such as RAG, persistent state and API access can be combined with other architectures.
| Dimension | Rule-based chatbot | Answer-only LLM chatbot | AI agent |
|---|---|---|---|
| Core engine | Intent classifier and predefined logic | LLM with a retrieval-and-answer workflow | LLM selects steps in a controlled loop |
| Response generation | Templates and stored values | Generated from supplied context | Generated from context and available tool results |
| Conversation flow | Defined transitions, including multi-turn state | Answer or clarify within the configured workflow | Model selects the next permitted step |
| Tool use | API calls can be built into the flow | No action tools in this example | Model proposes calls; application enforces access |
| Multi-step tasks | Explicit sequence and branches | Needs an added workflow for external operations | Can select a sequence within limits |
| Handling novel requests | Classifier and fallback coverage determine behavior | Answer only when supported by context | May choose a useful path or make an incorrect decision |
| Knowledge access | Configured search, APIs or stored responses | Often RAG; implementation-dependent | Often RAG and tools; implementation-dependent |
| Maintenance model | Maintain flows, templates, integrations and tests | Maintain sources, retrieval and evaluation | Maintain sources, tools, policies and evaluation |
| Cost structure | Hosting, integrations, licenses and human work | Platform or model usage plus human work | Platform or model usage, tools and human work |
| Failure mode | Misrouting, wrong state or API error | Unsupported answer or retrieval failure | Unsupported answer, wrong tool use or uncertain write |
| Latency | Depends on workflow and downstream services | Depends on retrieval, model and delivery | Depends on selected steps, services and delivery |
Do rule-based chatbots still make sense?
Yes. Choose the simplest architecture that meets the task’s quality, control and response-time requirements. A scripted flow may be enough for a stable form, a small set of approved answers or an authorized lookup with fixed steps. Adding a model does not remove the need to maintain integrations and policies.
Several conditions can favor predefined workflows:
Responses that must use approved wording. Templates can preserve reviewed text. That is one control, not proof of regulatory compliance; eligibility, disclosures, access and review obligations still depend on the deployment.
Simple, repeatable operations. A known sequence can avoid unnecessary model calls at any traffic volume. Compare actual hosting, licensing, API and maintenance costs rather than assuming a universal volume threshold.
Existing integrations that already work. Keep reliable workflows and add language understanding or selected tools where evaluation demonstrates a useful improvement. Replacing all intents and flows is not a prerequisite for using an LLM.
An agentic design is worth evaluating when the required steps vary and model-directed selection adds value. Use representative tasks and failure cases to establish that value; the architecture name supplies no default win rate.
Where AI agents are strongest
Model-directed orchestration is useful to evaluate when requests vary enough that selecting the next step at runtime could help. The following are candidate uses, not measured superiority claims:
Support requests that span systems. The return example combines policy retrieval, an authorized order read and a separately authorized request. The model may help choose the next relevant step, while a fixed workflow remains a valid alternative.
Open-ended product questions. A question about team size, channel access and plan limits may require combining several documented facts. Retrieval can help both an answer-only chatbot and an agent. Neither should infer legal compliance or undocumented entitlements from a product description.
Sales conversations. A model can collect stated requirements and retrieve permitted CRM context. Compare recommendation accuracy, consent and useful outcomes against the existing workflow before claiming higher effectiveness.
Multi-language support. Language models can reduce the need for separately authored responses, but supported languages and quality vary. Evaluate terminology, tone, policy fidelity and escalation in each target language. An agentic loop is not required for multilingual answers and does not eliminate localization work.
Escalation decisions. Configured rules and model judgments can help identify missing knowledge or requests for a person. Model confidence or sentiment is not reliable proof that the decision is correct. Verify the handoff request and staffing process separately from the AI reply.
For background on platform components, see the Conversational AI Platform Guide.
How AI agents work in production
Production systems combine model behavior with application controls. Similar controls can also support scripted workflows; they are not exclusive to agents.
Guardrails and behavioral rules. Instructions define scope, tone and escalation behavior. Application checks constrain permitted operations and stopping conditions. Test refusals and failures as well as expected requests; a prompt is not a guarantee that every answer will follow policy.
Knowledge grounding (RAG). Retrieval supplies relevant source material before generation and can reduce unsupported answers. It does not guarantee factual correctness. Inspect the retrieved source and the resulting claim, using the traceability the implementation actually exposes. See What are AI Hallucinations? for further context.
Tool access and action permissions. APIs or MCP servers can expose reads and writes. Limit credentials and allowed operations, enforce customer access at the destination, and treat returned text as data. A configured tool or a model’s proposed parameters cannot grant authority to another customer’s order or an unapproved refund.
Quickchat AI Agents supports Knowledge Base answers and configured API and MCP Actions. API Actions can use Run only when conditions checked outside the model; gate on trusted metadata and retain the destination API’s authorization. Human Handoff pauses AI replies and puts the conversation in the Inbox, assigned when routing selects a teammate or otherwise unassigned. Assignment does not guarantee an immediate human reply. The return workflow above would require suitable external services and configuration; it is not a promise of native return or refund fulfillment.
Cost and performance trade-offs
The choice between chatbot and AI agent is partly an architecture decision and partly an economic one.
Chatbot economics
A scripted workflow can avoid LLM inference, but its marginal cost is not necessarily zero. Include hosting, platform licenses, downstream APIs, monitoring and any human handling. A simple lookup may be inexpensive with either architecture; measure the actual deployment.
Maintenance includes flows, templates, integrations and test coverage. Agentic systems add their own work on prompts, retrieval, tools and evaluation. Estimate both on the same task scope and period rather than assuming that one has a lower maintenance burden.
AI agent economics
Model inference and tool execution consume resources, but a platform’s bill may be based on credits, conversations, resolutions, seats or a subscription. The underlying token cost is not necessarily the customer’s billing unit.
Compare current terms directly: Quickchat AI pricing, Intercom pricing and Salesforce Agentforce pricing use product-specific units and conditions. Check the base fee, minimum commitment, included usage, overages and definition of a billable outcome. A conversation and a verified resolution are not interchangeable. The AI agent pricing models guide explains how to compare those scopes without treating advertised rates as equivalent.
Use your own eligible-request cohort to estimate cost per verified outcome. Include failed attempts, repeat contacts, human rework, ongoing spend and one-time setup. Human-support cost also depends on channel and cost scope; a broad industry figure is not a matched baseline for your deployment. Released staff time is capacity, not cash savings unless an actual cost falls or a planned expense is avoided.
For a same-period calculation, see How to Calculate Chatbot ROI. No resolution-rate advantage or savings percentage is assumed here. Measure time to the first useful response and time to verified completion, including downstream services and handoff; neither architecture has a universal latency range.
Measuring success differently
Use common outcome criteria when comparing architectures, then add diagnostics for their different failure modes.
Workflow diagnostics include recognized intents, fallback reasons, state transitions, API errors and completion of required steps. A completed flow is not necessarily a resolved customer issue.
Outcome measures include verified resolution among eligible requests, repeat contacts, human rework, satisfaction with its response count, and cost per verified outcome. For an agent, also inspect unsupported claims, tool selection, access denials and uncertain writes. Do not count a generated answer, an accepted return request and a completed refund as the same outcome.
The Chatbot Analytics guide separates native product observations from measures that need external evidence and includes a synthetic worksheet. Define the denominator and observation window before comparing results; the worksheet is not a benchmark.
Keep enough evidence to audit a policy answer, an authorized lookup, an external mutation and a handoff separately. Retrieved sources, tool responses and authoritative system state serve different purposes. Use synthetic or appropriately protected records for evaluation, and do not expose private customer data in shared examples.
FAQ
Can I convert my existing chatbot into an AI agent?
You can often add language understanding or selected tools incrementally while retaining useful workflows, knowledge and integrations. Model-directed tool selection is an architectural change that needs evaluation. Keep authorization, required approvals and reliable deterministic steps rather than discarding them merely to use the agent label.
Are AI agents more expensive than chatbots?
There is no universal answer. Compare subscriptions, billing units, usage, setup, maintenance and human rework for the same eligible tasks and period. Model calls add resource costs, but retail billing varies. Higher resolution rates and cash savings must be measured; staff time released is not automatically money saved.
Do AI agents hallucinate?
Yes. Retrieval can provide relevant sources but does not guarantee a correct answer. Check whether claims follow from those sources and whether a tool result actually confirms the stated outcome. Apply the same scrutiny to answer-only LLM chatbots, and hand off when the evidence is insufficient.
Can an AI agent handle tasks a chatbot can’t?
An agent may select useful steps for varied requests that an existing workflow does not cover. Scripted chatbots can also call APIs, retain state and complete multi-step tasks. Actual capability depends on integrations, permissions, policy and evidence, not the interface label. A return request still requires separate write authority.
What about latency?
Measure time to first useful response and time to verified completion on representative tasks, including slow and failed downstream calls. Report distributions such as median and 95th percentile against your requirement. More model or tool steps can add delay, but neither label establishes a universal response time or makes that delay acceptable.
Is “AI agent” just a marketing rebrand of “chatbot”?
The labels overlap. A chatbot describes an interaction surface, while agentic orchestration usually means a model selects steps or tools at runtime. Inspect the actual workflow, authority checks, stopping conditions and outcome evidence. A product name alone does not establish greater autonomy, accuracy or business value.