I shipped a production AI agent that silently failed 17% of the time. The agent was designed to process user requests from Telegram, route them to a specialized Groq-powered summarizer, then to a Claude-powered content generator, and finally format the output for WhatsApp. My unit tests passed. My integration tests passed. My end-to-end tests, which checked the final output format, also passed. The problem was not if it produced output, but what output.
The agent was supposed to summarize a technical document and then generate a social media post. Without an AI agent evaluation harness, I would have continued shipping a system that generated social media posts based on irrelevant sections of the document, or worse, hallucinated key facts. This wasn't a bug in my Python code; it was a failure in the AI's reasoning, its ability to follow complex, multi-step instructions, and its robustness to varied inputs. This is why I stopped all feature development and spent two weeks building an AI agent evaluation harness with 131 tests across four distinct layers, costing me $0.03 per full run.
The Silent Failure: Why Unit Tests Are Blind
My initial test suite for the multi-agent system included:
- Unit tests: Checked individual Python functions, e.g.,
parse_telegram_input(),format_whatsapp_output(). (100% coverage) - Integration tests: Verified API calls to Groq and Claude, ensuring correct request/response structure. (Passed)
- End-to-end tests: Confirmed the final WhatsApp message structure and presence of expected keywords. (Passed)
The agent’s core task was:
1. Receive a document URL and a target social media platform from Telegram.
2. Fetch the document.
3. Summarize the document using Groq (for speed).
4. Generate a social media post using Claude (for nuance and creativity).
5. Send the post to WhatsApp.
The silent failure occurred in step 3 and 4. A user uploaded a PDF of a research paper. The Groq summarizer, under certain prompt variations, would focus on the "Acknowledgements" section or the "References" instead of the "Abstract" and "Methodology." Claude, receiving this skewed summary, would then generate a social media post about the authors' funding sources or related works, not the paper's findings. The final WhatsApp message looked correct structurally, but the content was fundamentally wrong.
This is a class of failure that traditional software tests cannot catch. Unit tests verify code logic. Integration tests verify component communication. End-to-end tests verify system flow and final output format. None of them verify semantic correctness or reasoning fidelity of an AI agent. This requires a dedicated AI agent evaluation harness.
Layer 1: Input Robustness (35 Tests)
The first layer of my evaluation harness focuses on how well the agent handles diverse and challenging inputs. My agents operate on Telegram and WhatsApp, meaning inputs are often unstructured, misspelled, or incomplete.
I built 35 test cases covering:
- Malformed URLs:
htps://example.com,example.com,ftp://example.com/doc.pdf. Expected: graceful error, prompt for correct URL. - Non-existent URLs:
https://nonexistent-domain-12345.com/doc.pdf. Expected: network error handling, user notification. - Unsupported document types:
https://example.com/image.jpg,https://example.com/video.mp4. Expected: "unsupported format" message. - Empty messages: User sends nothing. Expected: prompt for input.
- Gibberish: "asdfghjkl;" Expected: "I don't understand" message.
- Language variations: Requests in Spanish, French (my agents are English-only for now). Expected: "English only" message.
- Long inputs: A 10,000-word text pasted directly into Telegram. Expected: truncation or "too long" message.
Each test case is a JSON object defining the input, expected output (regex or specific string), and a pass/fail condition. The harness simulates a Telegram message, injects it into the agent's entry point, and captures the final response. This layer alone caught 8 critical failures related to URL parsing and content fetching that my previous tests missed. For example, a malformed URL like example.com would previously crash the document fetching service, leading to a silent failure for the user. Now, it triggers a specific "invalid URL" response.
Layer 2: Core Reasoning & Prompt Adherence (50 Tests)
This is the most critical layer, designed to catch the semantic failures I described. It focuses on the agent's ability to understand and execute complex instructions, especially across multiple LLM calls.
I developed 50 test cases, each with:
- A specific document: (e.g., a research paper, a news article, a product manual).
- A specific prompt/instruction: (e.g., "Summarize this for a 10-year-old," "Extract key takeaways for investors," "Generate a LinkedIn post highlighting the challenges.")
- Evaluation criteria: These are not simple string matches. They are a combination of:
* Keyword presence/absence: "Must contain 'AI' and 'Panama', must NOT contain 'Acknowledgements'."
* Semantic similarity: Using an embedding model (e.g.,
text-embedding-ada-002 via Oracle OCI Generative AI service) to compare the generated output against a human-written "gold standard" answer. A cosine similarity score below 0.7 triggers a failure.* Factuality check: For specific factual extraction tasks, I use a small, fine-tuned LLM (running on Oracle Cloud Infrastructure's GPU instances) to act as a "critic," comparing generated facts against the source document. This critic model is prompted with "Given the document X, is statement Y true? Answer only 'Yes' or 'No'."
* Constraint adherence: "Output must be under 280 characters," "Must use bullet points."
Example test:
- Document: A 5-page PDF on "Quantum Computing Advancements."
- Prompt: "Generate a tweet summarizing the main breakthrough for a general audience. Keep it under 280 characters. Focus on the practical implications."
- Evaluation:
* Length check:
< 280 chars.* Keyword check:
contains "quantum", contains "breakthrough", NOT contains "Schrödinger equation".* Semantic similarity: Compare generated tweet embedding to a human-written tweet embedding (cosine similarity > 0.75).
* Factuality: Critic model checks if the "practical implications" mentioned are actually present in the document.
This layer revealed that my Groq summarizer, while fast, sometimes over-simplified or omitted crucial context, leading Claude to generate misleading posts. I adjusted Groq's system prompt to explicitly emphasize "main findings and their implications" and added a post-processing step to re-rank summary sentences based on their embedding similarity to the initial user query.
Layer 3: Multi-Turn & State Management (26 Tests)
My agents are designed for conversational interfaces. This means they need to maintain context and respond appropriately in multi-turn interactions.
26 tests cover:
- Follow-up questions: User asks "What about X?" after a summary. Expected: agent refers to the previous summary and answers X if relevant.
- Clarification requests: Agent asks "Which document are you referring to?" if context is ambiguous.
- Context switching: User asks for a summary of Document A, then immediately asks for a social media post for Document B. Expected: agent correctly switches context and processes Document B.
- Interrupted flows: User starts a task, then sends "stop" or "cancel." Expected: agent gracefully terminates the current task.
- Error recovery: Agent encounters an error (e.g., API timeout), then user retries. Expected: agent attempts to restart or guide the user.
These tests simulate a sequence of messages, checking the agent's state at each step. For instance, after a user provides a document URL, the harness checks if the agent's internal state correctly stores this URL before the next message. This layer helped me refine my state management logic, moving from a simple dictionary to a more robust, session-based context store backed by Oracle Autonomous Database.
Layer 4: Performance & Cost (20 Tests)
While not directly about correctness, performance and cost are critical for production systems, especially with LLM APIs.
20 tests monitor:
- Latency: Time taken for specific operations (e.g., summarization, generation). I set thresholds (e.g., Groq summarization < 2 seconds, Claude generation < 10 seconds).
- Token usage: Number of input/output tokens for each LLM call. This directly impacts cost. I monitor for unexpected spikes.
- API call count: Number of external API calls per request. An unexpected increase could indicate a loop or inefficient prompting.
- Error rates: Percentage of API calls returning non-200 status codes.
- Total cost per run: The harness aggregates token usage and API calls, then calculates the estimated cost based on current provider rates (Groq: $0.0002/1K tokens, Claude 3 Sonnet: $3/M input tokens, $15/M output tokens). My target is $0.03 per full agent interaction.
The harness runs these 131 tests nightly on Oracle Cloud Infrastructure's always-free tier compute instances. The total cost for a full run, including LLM calls, is consistently around $0.03. This low cost allows me to run it frequently, catching regressions quickly. The performance tests revealed that certain complex prompts for Claude were pushing generation times beyond acceptable limits, leading me to optimize prompt structure and explore smaller Claude models for specific tasks.
The Cost of Not Evaluating
Building this AI agent evaluation harness took me two weeks. This was time I wasn't building new features, wasn't acquiring new users. But without it, I would have continued shipping a product that was fundamentally broken for a significant portion of its use cases. The cost of debugging production issues, losing user trust, and rebuilding features would have been far higher.
My 131 tests across four layers provide a safety net that traditional software testing cannot. They ensure my multi-agent system not only functions technically but also reasons correctly, adheres to instructions, and provides valuable, accurate outputs. This is non-negotiable for any AI agent moving into production.
Frequently Asked Questions
Q: How do you manage the "gold standard" answers for semantic similarity tests?
A: For critical paths, I manually create 2-3 "gold standard" answers per test case. For less critical paths, I use a stronger LLM (like Claude 3 Opus) to generate a reference answer, which is then manually reviewed and approved.
Q: What if the LLM models change or update, invalidating my semantic similarity scores?
A: This is a real risk. I pin to specific model versions where possible (e.g., claude-3-sonnet-20240229). When models update, I re-run the entire eval harness. If a significant number of semantic similarity tests fail, it indicates a model drift, and I either adjust the gold standards or fine-tune my prompts.
Q: How do you handle the cost of running 131 tests, especially with expensive LLMs?
A: I optimize by using cheaper, faster models (like Groq) for summarization and initial routing within the harness itself where possible. For the actual generation tests, I use the production models but keep the input documents and desired outputs concise to minimize token usage. The $0.03/run is a calculated average, and I monitor it closely.
Q: What's your strategy for evaluating agents that generate creative content, where there's no single "correct" answer?
A: For creative content, I rely more on constraint adherence (e.g., tone, style, length) and negative constraints (e.g., "must not be offensive," "must not hallucinate facts"). Semantic similarity is used to check for relevance to the prompt, not exact phrasing. Human review of a subset of creative outputs is also essential.
Q: How do you integrate this harness into your CI/CD pipeline on Oracle Cloud?
A: The harness is a separate Python application. I use Oracle Cloud Infrastructure (OCI) DevOps to trigger a nightly run. The results (pass/fail, latency, cost) are stored in an OCI Object Storage bucket and visualized via OCI Logging Analytics dashboards. A critical failure triggers a notification via OCI Notifications.