How much does it cost to run a production AI chatbot per month?

For a small business chatbot handling 200-500 conversations per month, expect $15-40 in API costs using a model like GPT-4o-mini or Claude Haiku. The harness infrastructure (hosting, logging, email relay) adds another $5-10 if self-hosted on a small VPS.

Should I use a single model or route between multiple models?

Route. Use a fast, cheap model (GPT-4o-mini, Claude Haiku) for routine queries and escalate to a larger model only when the smaller one signals low confidence or when the query matches a complexity trigger. This typically cuts costs by 60-70% without sacrificing quality.

How do I prevent my chatbot from making commitments the business cannot honour?

Constrain the system prompt with explicit boundaries: list what the chatbot can and cannot promise. Add a post-processing step that flags any response containing pricing, legal, or delivery commitments for human review before sending.

What is the most common failure mode in production chatbots?

Hallucinated contact details. The model invents phone numbers, email addresses, or office hours that do not exist. The fix is to ground all factual claims in a structured data source injected into the context, never relying on the model's training data for business-specific information.

Do I need a vector database for a business chatbot?

Usually no. For most small business chatbots, the knowledge base fits entirely in the context window. Inject the full FAQ, pricing table, and service descriptions as structured text. A vector store adds complexity and latency without meaningful benefit until you have thousands of documents.

Harnesses

Harness Engineering Lessons from Production AI Chatbots

Real technical lessons from deploying AI chatbots for client websites: prompt architecture, context management, fallback chains, cost control, and the harness patterns that actually matter in production.

14 September 2026 12 min read

Every AI chatbot tutorial shows you the happy path: send a message, get a sensible reply, ship it. Production is not the happy path. Production is a customer asking for a refund on a product you stopped selling six months ago, a competitor's name appearing in your chatbot's responses, and a model that confidently provides a phone number that has never belonged to your business.

Here is what we learned building this at Daedalus Design across a dozen production deployments for small business clients. These are not theoretical patterns. They are the harness engineering decisions that separated the chatbots that worked from the ones we had to rebuild.

1. System prompt architecture matters more than model choice

The single biggest lever for chatbot quality is not which model you use. It is how you structure the system prompt. A well-architected system prompt on GPT-4o-mini will outperform a lazy prompt on Claude Opus every time, at a fraction of the cost.

The architecture that works in production has four layers:

Layer 1: Identity and boundaries
  - Who the chatbot is, what business it represents
  - Explicit "do not" constraints (no legal advice, no pricing guarantees)

Layer 2: Knowledge injection
  - Structured business data (services, pricing, hours, location)
  - FAQ entries as Q/A pairs
  - Current promotions or time-sensitive information

Layer 3: Behaviour rules
  - How to handle unknown questions (escalate, do not guess)
  - Tone and formality calibration
  - When to offer human handoff

Layer 4: Output formatting
  - Response length constraints
  - Whether to ask follow-up questions
  - How to end a conversation

The critical insight: each layer serves a different failure mode. Layer 1 prevents liability. Layer 2 prevents hallucination. Layer 3 prevents bad user experiences. Layer 4 prevents bloated, unfocused responses. If you collapse all four into a single paragraph, you lose the ability to diagnose which layer failed when something goes wrong.

2. Ground everything in structured data

The most common failure mode in production chatbots is not toxic output or jailbreaks. It is hallucinated business details. The model invents phone numbers, email addresses, office hours, or pricing that do not exist. This happens because the model is drawing on its training data rather than your actual business information.

The fix is simple but non-negotiable: inject all factual business data as structured text in the context. Do not rely on the model to "know" your business. Here is the pattern:

<business_facts>
Name: Acme Plumbing Ltd
Phone: 0117 496 1234
Email: hello@acmeplumbing.co.uk
Hours: Mon-Fri 08:00-18:00, Sat 09:00-13:00
Emergency rate: £95 call-out + £45/hr
Standard rate: £65 call-out + £35/hr
Service area: Bristol, Bath, North Somerset
NOT available: Gas work (Gas Safe registered plumber handles this separately)
</business_facts>

This block goes in every request, not just the first message of a conversation. Models can drift over long conversations, and re-injecting the grounding data on every turn prevents that drift from compounding.

Why vector databases are usually overkill

A common instinct is to reach for a vector database (Pinecone, Weaviate, Chroma) to store business knowledge. For most small business chatbots, this is unnecessary complexity. The entire knowledge base — FAQ, pricing, service descriptions, contact details — fits comfortably in 2,000-4,000 tokens of structured text. That is well within the context window of every current model.

Vector stores make sense when you have thousands of documents, product catalogues, or technical manuals. For a chatbot that needs to answer questions about a plumbing business, a law firm, or a dental practice, structured text injection is simpler, faster, cheaper, and more reliable.

3. Model routing for cost control

Running every query through a frontier model is expensive and unnecessary. Most customer queries fall into predictable patterns: "What are your hours?", "Do you cover my area?", "How much does X cost?" A small, fast model handles these perfectly.

The routing pattern that works:

Step 1: Send query to fast model (GPT-4o-mini, Claude Haiku)
Step 2: Check confidence signal
  - If model responds normally → return response
  - If model says "I'm not sure" or "I don't have that information" → escalate
  - If query contains complexity triggers (complaints, multi-part
    questions, technical details) → escalate
Step 3: Escalated query goes to larger model (GPT-4o, Claude Sonnet)
Step 4: Larger model response includes [ESCALATED] tag in logs

In practice, this routing cuts costs by 60-70%. A chatbot handling 400 conversations per month might cost £3-5 with routing versus £12-18 without it. The quality difference is negligible for routine queries because routine queries do not require frontier intelligence.

The confidence signal does not need to be sophisticated. A simple instruction in the system prompt — "If you are not confident in your answer, respond with exactly: ESCALATE_TO_HUMAN" — works reliably across current models. Parse the response for that string before sending it to the customer.

4. Fallback chains, not fallback messages

Most chatbot implementations have a single fallback: "I'm sorry, I can't help with that. Would you like to speak to someone?" This is a dead end. The customer either leaves or calls, and you have lost the value of having a chatbot at all.

A fallback chain provides a sequence of increasingly specific alternatives:

Fallback 1: Rephrase the question and try the knowledge base again
Fallback 2: Offer related topics the chatbot CAN help with
Fallback 3: Provide the business's direct contact details
Fallback 4: Offer to take a message (name, email, brief description)
Fallback 5: Escalate to live human if available

Each fallback level has a purpose. Level 1 catches cases where the customer used unusual phrasing. Level 2 redirects rather than shutting down. Level 3 ensures the customer can always reach a human. Level 4 captures the lead even when no human is available. Level 5 handles urgent cases.

The implementation is a simple state machine in the harness code. The model does not need to manage the fallback chain — the harness does. The model just needs to signal when it cannot answer, and the harness takes over from there.

5. Conversation logging is not optional

If you deploy a chatbot without logging every conversation, you are flying blind. You cannot improve what you cannot measure. You cannot debug what you cannot review. You cannot defend your chatbot's behaviour if you have no record of what it said.

The minimum viable logging setup:

{
  "conversation_id": "uuid",
  "started_at": "2026-09-14T10:23:00Z",
  "messages": [
    {"role": "user", "content": "...", "timestamp": "..."},
    {"role": "assistant", "content": "...", "timestamp": "...", "model": "gpt-4o-mini", "tokens": 142}
  ],
  "escalated": false,
  "resolved": true,
  "total_cost_pence": 0.3
}

Log the model used, the token count, and the cost. This data feeds back into your routing decisions. If you see that a particular type of query always escalates, you can add it to the fast model's knowledge base. If a particular response pattern generates complaints, you can adjust the system prompt.

The cost tracking per conversation is essential for client reporting. Business owners want to know what the chatbot costs per enquiry, not just per month. The per-conversation cost also lets you identify anomalously expensive conversations — usually caused by the model getting into a loop or the context growing too large.

6. The static site problem

A specific challenge we encountered repeatedly: clients with static HTML sites who wanted an AI chatbot. Static sites have no server-side logic, no database, no backend API. The chatbot has to run entirely in the browser or through an external service.

The architecture that works: a lightweight JavaScript widget that connects to an external API endpoint. The widget handles the UI (chat window, message input, typing indicators). The API endpoint handles the model calls, knowledge injection, conversation logging, and email relay for lead capture.

[Browser] --> [API Gateway] --> [Harness: prompt assembly + model call]
                                      |
                                      +--> [Logging store]
                                      +--> [Email relay for leads]
                                      +--> [Analytics]

The API gateway can be a simple serverless function (Cloudflare Workers, AWS Lambda) or a lightweight Node.js service on a £5/month VPS. The key constraint is that the business data (FAQ, pricing, hours) must be injected server-side, not client-side. If you inject it client-side, the customer can see your full knowledge base in the browser's developer tools, including any internal notes or competitor comparisons you included for the model's context.

What this means for harness engineering

Production AI chatbots are a harness engineering problem, not a model problem. The model provides the language capability. The harness provides everything else: prompt architecture, data grounding, cost control, fallback handling, logging, and the integration points that make the chatbot useful to the business.

The patterns above are not specific to chatbots. They apply to any production AI system: document summarizers, email classifiers, content generators. The harness is where the engineering value lives. The model is a commodity. The harness is the product.