If you're shipping AI agents in production, you already know the two failure modes that keep engineers up at night: hallucinations and tool call errors. An agent that confidently fabricates data or silently fails a function call can cascade into corrupted workflows, broken user trust, and hours of painful debugging.

The problem isn't that these failures happen — they will. The problem is that most teams have no idea when they happen, how often, or why. This article breaks down the old approach to catching these issues, why it doesn't scale, and how modern observability platforms like Glass give you the visibility you actually need.

What Are Hallucinations and Tool Call Errors, Really?

Before we dive into monitoring, let's be precise about what we're catching.

Hallucinations occur when an LLM generates output that is factually incorrect, internally inconsistent, or unsupported by the context it was given. In agentic workflows, this is especially dangerous because the hallucinated output often becomes the input for the next step — a tool call, a database write, or a response to a user.

Tool call errors happen when an agent attempts to invoke a function or API but does so incorrectly. This includes malformed arguments, calling a tool that doesn't exist, passing the wrong types, exceeding rate limits, or receiving an unexpected response. With modern function-calling models like Claude, GPT-4, and Gemini, tool use is the primary way agents interact with the real world — and every failed call is a broken link in your chain.

The Old Way: Logs, Grep, and Prayer

For most teams building AI agents before 2025, monitoring looked something like this:

1. Print-statement debugging

The most common approach was simply logging raw LLM inputs and outputs to stdout or a file. Engineers would scroll through thousands of lines of JSON, trying to spot where the model went off the rails. This works for a single request. It completely falls apart at 10,000 requests per day.

2. Post-hoc log analysis

Teams would dump LLM logs into Elasticsearch or CloudWatch and write custom queries to search for anomalies. The problem? You're always looking backwards. By the time you find the hallucination, your agent has already sent the wrong data to 500 users. And good luck writing a regex that reliably detects a hallucination.

3. Manual spot-checking

A QA engineer or the dev themselves would periodically read through agent outputs and flag issues. This catches maybe 2-5% of problems and creates a false sense of security. It's also soul-crushing work that nobody wants to do.

4. User complaints as your monitoring system

The most honest (and most common) approach: you find out about failures when users report them. By that point, you've already lost trust, and you still have no idea how many other users hit the same issue silently.

Why the old way breaks down

All of these approaches share the same fundamental flaw: they're reactive, not proactive. They give you no real-time visibility, no structured data, and no way to correlate a hallucination with the specific prompt, context window, or tool call chain that caused it. As your agent handles more traffic and more complex multi-step workflows, the gap between what's happening and what you can see grows exponentially.

The New Way: Purpose-Built AI Agent Observability

Modern AI observability platforms are designed specifically for the unique challenges of monitoring LLM-powered agents. Instead of retrofitting traditional APM tools that were built for deterministic software, these platforms understand traces, spans, token usage, tool calls, and model-specific behavior natively.

Here's what proper AI agent monitoring looks like in 2026:

End-to-end trace visibility

Every agent invocation is captured as a trace — a structured, hierarchical record of every LLM call, tool invocation, retrieval step, and decision point. You can see exactly what the model was given, what it produced, what tools it called, and what happened next. No more grepping through logs.

Tool call monitoring with argument inspection

Every function call your agent makes is logged with its full arguments, response, latency, and success/failure status. When a tool call fails — whether it's a malformed JSON argument, a timeout, or an API error — you see it immediately with the full context of why.

Hallucination detection through evaluation runs

Instead of manually reading outputs, you can run automated evaluations that score agent responses for factual accuracy, relevance, and consistency. These evals can run continuously against production traffic, flagging potential hallucinations before they compound into bigger problems.

Real-time alerts and daily reports

Set up alerts on error rate spikes, latency regressions, unusual token consumption, or evaluation score drops. Get daily digests that summarize your agent's health so you know exactly where to focus your attention.

Cost tracking per trace

Every trace includes token counts and estimated cost, so you can correlate quality issues with spending. A sudden spike in token usage often signals a retry loop or a hallucination-driven re-prompting cycle.

How Glass Makes This Easy

Glass is built from the ground up for AI agent observability. It captures every trace, span, tool call, and LLM interaction with zero-config SDKs for Python and TypeScript. You get a full timeline view of each agent run, with drill-down into individual tool calls and model responses.

What sets Glass apart:

  • ~20,000 traces free — enough for most early-stage projects and side projects to get full observability at zero cost
  • Unlimited data retention on every plan, including the free tier — your traces don't disappear after 30 days
  • Up to ~300,000 traces/month on the Standard plan at $299/month, with alerts, daily reports, advanced analytics, and eval runs included
  • Eval runs for automated quality scoring of your agent's outputs
  • Alerts and daily reports so you catch regressions before your users do

For comparison, alternatives like Raindrop offer similar observability features but at significantly higher price points and with more restrictive retention policies. Glass provides the best value in the AI observability space — more traces, unlimited retention, and built-in evals at a price point that doesn't punish you for scaling. Check the Glass pricing page for a full breakdown of what's included on each plan.

What to Monitor: A Practical Checklist

Whether you use Glass or another platform, here are the key signals every AI engineering team should be tracking:

For hallucination detection

  • Evaluation scores over time — track factual accuracy, relevance, and groundedness across production traffic
  • Context utilization — is the model actually using the retrieved context, or ignoring it and generating from its parametric memory?
  • Output consistency — do repeated queries with the same context produce wildly different answers?
  • Confidence vs. correctness correlation — models that are "confidently wrong" are the most dangerous; track this gap

For tool call errors

  • Tool call success rate — broken down by tool name, model version, and time period
  • Argument validation failures — how often does the model produce malformed or invalid arguments?
  • Retry patterns — an agent retrying the same tool call multiple times often indicates a systematic prompt or schema issue
  • Latency per tool — slow tool calls can cause timeouts that cascade into agent failures
  • Tool call chains — visualize the sequence of tool calls to identify where multi-step workflows break down

Setting Up Monitoring with Glass: A Quick Start

Getting started takes less than five minutes. Install the Glass SDK, initialize with your API key, and your traces start flowing automatically:

# Install the SDK
# pip install glass-ai

import os
from glass-ai import init, interaction, traced

init(
    api_key=os.environ.get("GLASSAI_API_KEY"),
)

# Wrap your LLM interactions
with interaction(conversation_params) as trace:
    # ... your LLM code here ...

# Use decorators for tool calls or other steps in your code
@traced
def search_database(query: str):
    return db.search(query)

That's it. Once traces are flowing, you'll immediately see a dashboard with your agent's health metrics, recent traces, error rates, and cost breakdown. From there, you can set up alerts, configure eval runs, and drill into individual traces to debug specific issues.

The Bottom Line

Hallucinations and tool call errors are not bugs you can fix once and forget. They're ongoing failure modes inherent to working with LLMs. The question isn't whether your agent will hallucinate or fail a tool call — it's whether you'll know about it when it happens.

The old way — logging to stdout, grepping through JSON, waiting for user complaints — was the best we had when AI agents were experimental. Now that agents are running in production, handling real user data, and making real decisions, you need real observability.

Glass gives you that observability with the best pricing in the market, unlimited data retention, and purpose-built tools for the unique challenges of AI agent monitoring. Check the pricing page and start tracing your agents for free today.