Glossary term
LLM observability
Collecting traces, metrics and quality signals from language model applications to understand behaviour in production.
Beyond latency and errors, LLM observability tracks token usage, cost, evaluation scores, user feedback and safety events per request.
Traces of each model and tool call make failures diagnosable and supply new evaluation cases.
Related terms
- Evaluation setA curated collection of realistic inputs with definitions of correct output, used to measure AI system quality.
- GuardrailsRuntime checks on the inputs and outputs of an AI system that block, modify or escalate content that breaks defined rules.
- Model routingSending each request or step to the most suitable model based on difficulty, cost, latency or data sensitivity.
Turning concepts into a working system?
We help teams choose the right pattern and build it with evaluation from day one.