Most AI chatbots feel like talking to a very polite wall. You ask a question, it gives you a generic answer, and if you need something specific, you’re stuck in a loop. We’ve all been there. The problem is that we try to make one single LLM prompt do everything. We ask it to be a billing expert, a technical guru, and a sales closer all at once. It’s too much. It gets confused. It hallucinates.
The fix is to stop building a bot and start building a team. I’m talking about a multi-agent AI workflow tutorial where we split the labor. One agent handles the greeting and routing, while specialized agents handle the actual work. It’s a lot more stable.
Designing the Triage Logic
The triage agent is basically your digital receptionist. Its only job is to figure out what the user wants and send them to the right place. If you give this agent too many instructions, it starts overthinking. I learned this the hard way last year when I tried to build a support bot for a client using LangGraph. I gave the triage agent a 2,000-word manual on company policy. Result? It spent three minutes “thinking” before it finally decided the user wanted to talk to billing. It was a disaster.
Keep the triage logic lean. Use a simple classification prompt. Tell the agent: “You are a router. Identify if the user needs Billing, Tech Support, or Sales. If you aren’t sure, ask one clarifying question.”
You can use function calling (or tool use) to make this happen. Instead of the agent just saying “I’ll send you to billing,” it triggers a function called route_to_billing(). This makes the transition programmatic and less prone to the AI just chatting about how it’s going to help you without actually doing it.
Setting Up the Specialist Agents
Now we build the experts. Each specialist needs its own narrow system prompt and its own set of tools.
The Billing Agent
This one needs access to your payment API (like Stripe). It shouldn’t know how to troubleshoot a software bug. Its prompt should be strictly about invoices, refunds, and plan upgrades. If a user asks it about a technical bug, it should be programmed to send them back to triage.
The Tech Support Agent
This is where you connect your documentation. I usually hook this up to a vector database using RAG (Retrieval-Augmented Generation). I remember setting up a tech agent for a SaaS tool that had 400 different API endpoints. I initially used GPT-3.5 for the RAG retrieval, and it was honestly kind of a mess. It kept citing the wrong version of the docs. Switching to GPT-4o improved the accuracy from about 60% to 92% almost overnight, though it cost me a bit more in tokens.
The Sales Agent
The sales agent is different. It needs to be persuasive but not annoying. Give it a tool to check calendar availability or a way to push a lead directly into a CRM like HubSpot.
Implementing Shared Memory
This is the part where most people mess up. If a customer tells the triage agent their order number is #12345, and then they get moved to the billing agent, they shouldn’t have to say #12345 again. Nothing kills a user experience faster than “As I told the last bot…”
You need a shared state. In a multi-agent setup, you don’t just pass the last message. You pass a state object. This object contains the conversation history, user metadata, and any extracted entities (like that order number).
I’m still not entirely sure if using a Redis cache is the absolute best way to handle this for every use case, but it’s what we’ve found works for most. The agent reads the state, performs its task, and updates the state before handing it off. It’s basically a shared notebook that every agent can read and write to.
Testing the Hand-off
Testing this is tedious. You have to try and break it. I spend hours trying to trick my agents into looping. For example, I’ll tell the billing agent that my payment failed because of a technical bug in the app. If the billing agent tries to fix the bug instead of routing me back to tech support, the system is broken.
Check for these three things:
- Context Retention: Does the specialist know the user’s name and goal?
- Loop Prevention: Does the user get stuck bouncing between two agents?
- Fallback Logic: What happens when the triage agent can’t categorize the request?
Sometimes the agents just refuse to hand off. They get “too helpful” and try to answer everything themselves. When that happens, you have to tighten the system prompts. Be aggressive. Tell them: “You are NOT allowed to answer billing questions. You MUST route them.”
It’s a bit of a balancing act. Too strict and the bot feels robotic. Too loose and it becomes a jack of all trades, master of none.
Building this is a bit of a grind, but it’s the only way to get production-grade AI support. Once you get the routing and memory right, the rest is just tweaking prompts.
Talk soon,
StartMit Team
