Observe

Telemetry, metrics, and logging for production agent visibility.

Why Observability?

Defense without observability is flying blind. You need to know what is happening with your protected agents, not just that Guardrails are running, but what they are catching, how often, and what patterns are emerging.

Dome includes built-in telemetry that captures the full picture of agent behavior in production. This data serves three purposes:

1. Operational visibility: Are your Guardrails working? What is getting blocked? What is the latency impact?
2. Security monitoring: Are you seeing attack patterns? Anomalous behavior? Escalating threats?
3. Continuous improvement: What is happening in production that evaluation did not anticipate?

Telemetry

Dome automatically captures telemetry from every protected interaction. When instrumented by the developer, this includes:

Execution Traces

Full traces of LLM and tool execution:

{
  "trace_id": "abc123",
  "timestamp": "2024-01-15T10:30:00Z",
  "agent_id": "support-bot-v2",
  "spans": [
    {
      "name": "input_guardrail",
      "duration_ms": 23,
      "guards_executed": ["prompt-injection", "pii-detection"],
      "result": "pass"
    },
    {
      "name": "llm_call",
      "model": "gpt-4",
      "duration_ms": 1250,
      "tokens": {"input": 450, "output": 230}
    },
    {
      "name": "tool_call",
      "tool": "search_knowledge_base",
      "duration_ms": 89,
      "result": "success"
    },
    {
      "name": "output_guardrail",
      "duration_ms": 18,
      "guards_executed": ["toxicity", "pii-redaction"],
      "result": "pass"
    }
  ]
}

Traces let you see exactly what happened in each interaction—which Guards ran, what the LLM did, which tools were called, and how long each step took.

Instrumentation

To capture full traces, instrument your agent with Dome’s SDK:

from vijil.dome import DomeClient, trace

dome = DomeClient()

@trace("my-agent")
async def handle_request(user_input: str):
    # Input guardrail (automatic)
    filtered_input = await dome.scan_input(user_input)

    # LLM call (traced)
    with trace.span("llm_call"):
        response = await llm.generate(filtered_input)

    # Tool calls (traced)
    with trace.span("tool_call", tool="search"):
        results = await search_tool(response.query)

    # Output guardrail (automatic)
    filtered_output = await dome.scan_output(response.text)

    return filtered_output

Guardrail spans are captured automatically. LLM and tool spans require explicit instrumentation but provide complete visibility into agent behavior.

Metrics

Dome aggregates telemetry into operational metrics:

Protection Metrics

Metric	Description
inputs_scanned	Total inputs processed by input Guardrails
inputs_blocked	Inputs blocked by Guards
inputs_redacted	Inputs modified by redaction Guards
outputs_scanned	Total outputs processed by output Guardrails
outputs_blocked	Outputs blocked by Guards
outputs_redacted	Outputs modified by redaction Guards
block_rate	Percentage of interactions blocked

Latency Metrics

Metric	Description
input_guardrail_p50	Median input Guardrail latency
input_guardrail_p99	99th percentile input Guardrail latency
output_guardrail_p50	Median output Guardrail latency
output_guardrail_p99	99th percentile output Guardrail latency
total_overhead_ms	Total latency added by Dome

Guard-Level Metrics

Metric	Description
guard.{name}.triggers	How often this Guard triggers
guard.{name}.confidence_avg	Average confidence when triggered
guard.{name}.latency_p50	Median Guard execution time

Metrics are available via API, dashboard, and can be exported to your observability stack (Datadog, Prometheus, etc.).

Logs

Dome generates structured logs for every significant event:

Detection Logs

When a Guard detects a threat:

{
  "level": "warn",
  "event": "guard_triggered",
  "timestamp": "2024-01-15T10:30:00Z",
  "agent_id": "support-bot-v2",
  "guard": "prompt-injection",
  "action": "block",
  "confidence": 0.94,
  "input_hash": "sha256:abc...",
  "evidence": "Detected instruction override pattern"
}

Audit Logs

For compliance and security review:

{
  "level": "info",
  "event": "interaction_complete",
  "timestamp": "2024-01-15T10:30:02Z",
  "agent_id": "support-bot-v2",
  "session_id": "sess_xyz",
  "input_guardrail": {"result": "pass", "duration_ms": 23},
  "output_guardrail": {"result": "redact", "duration_ms": 18, "guard": "pii"},
  "total_duration_ms": 1380
}

Logs can be shipped to your SIEM, log aggregator, or stored in Vijil for analysis.

Dashboard

The Dome dashboard provides real-time visibility:

• Overview: Block rates, latency, active agents
• Guard performance: Which Guards are triggering, at what rates
• Threat timeline: Detection events over time
• Drill-down: From metric to trace to individual interaction

Next Steps

Observe Traces

Monitor Guardrail performance

How Defense Works

The defense architecture