Contact

LOG IN

Sign up

• ko
• ja
• en
• de

Large language models like GPT-4o and LLaMA are powering a new wave of AI applications, from chatbots and coding assistants to research tools. However, deploying these LLM-powered applications in production is far more challenging than traditional software or even typical machine learning systems.LLMs are massive and non-deterministic, often behaving as black boxes with unpredictable outputs. Issues such as false or biased answers can arise unexpectedly, and performance or cost can spiral if not managed. This is where LLM observability comes in.In this article, we will explain what LLM observability is and why it matters for managing LLM applications. We will explore common problems like hallucinations and prompt injection, distinguish observability from standard monitoring, and discuss the key challenges in debugging LLM systems. We will also highlight critical features to look for in LLM observability tools and survey the capabilities of current solutions. Finally, we will walk through a simple tutorial using W&B Weave to track an LLM’s outputs, detect anomalies such as hallucinations or bias, and visualize metrics. By the end, you will understand how LLM observability can enhance the reliability, performance, and trustworthiness of your LLM-driven applications. 

What is LLM observability?

 LLM observability refers to the tools, practices, and infrastructure that give you visibility into every aspect of an LLM application’s behavior – from its technical performance (like latency or errors) to the quality of the content it generates. In simpler terms, it means having the ability to monitor, trace, and analyze how your LLM system is functioning and why it produces the outputs that it does.Unlike basic monitoring that might only track system metrics, LLM observability goes deeper to evaluate whether the model’s outputs are useful, accurate, and safe. It creates a feedback loop where raw data from the model (prompts, responses, internal metrics) is turned into actionable insights for developers and ML engineers.This observability is crucial for several reasons:First, running LLMs in production demands continuous oversight due to their complexity and unpredictability. Proper observability ensures the model is producing clean, high-quality outputs and allows teams to catch issues like inaccuracies or offensive content early. It helps mitigate hallucinations (made-up facts) by flagging questionable answers, and it guards against prompt injection attacks or other misuse by monitoring inputs and outputs for anomalies.Observability is also key to managing performance – you can track response times, throughput, and resource usage in real time to prevent latency spikes or outages. It aids in cost management by monitoring token consumption and API usage, so you are not surprised by an exorbitant bill.Moreover, strong observability supports secure and ethical deployments: by detecting bias or privacy leaks in outputs, and by providing audit trails, it helps ensure the LLM is used in a compliant and trustworthy manner. In short, LLM observability gives you the confidence to operate LLM applications reliably at scale, knowing you can spot and fix problems before they harm the user experience or the business.

Common issues in LLM applications

Even advanced LLMs can exhibit a variety of issues when deployed. Below are some of the common problems that necessitate careful observability:

• Hallucinations: LLMs sometimes generate information that is factually incorrect or entirely fabricated, despite sounding confident. This phenomenon, known as hallucination, often occurs when the model is confronted with queries outside its knowledge scope. Instead of admitting it doesn’t know, the model may produce a plausible-sounding but false answer. Hallucinations can mislead users and propagate misinformation, so it’s critical to catch and correct them – especially in domains requiring factual accuracy (medicine, finance, etc.).

• Prompt injection attacks: Also called prompt hacking, this is a security issue where a user intentionally crafts an input that manipulates the LLM into deviating from its intended behavior. For example, a user might try to trick a chatbot into revealing confidential information or producing disallowed content by injecting malicious instructions in a prompt. Without safeguards, prompt injection can cause an LLM to output harmful or unauthorized material. Observability can help detect such attacks by analyzing input patterns and the model’s responses for signs of manipulation.

• Latency and performance bottlenecks: LLM-powered applications can suffer from high latency, especially if using very large models or multiple chained calls. Users might experience slow responses if the model or its pipeline isn’t optimized. Performance issues can stem from external API delays, heavy computation, or inefficient prompt designs. Furthermore, as usage scales, the system might hit throughput limits. For instance, a model could handle single requests quickly but choke when many queries arrive concurrently. Continuous monitoring of latency and throughput metrics is necessary to identify bottlenecks and maintain a smooth user experience.

• Cost unpredictability: Many LLMs incur cost based on token usage (the amount of text processed). Applications may face unpredictable or rising costs if prompts or outputs become longer than expected or if user traffic spikes. A slight change in how users interact (e.g. asking for more detailed answers) can multiply token consumption. Without observability, these cost drivers might go unnoticed until a bill arrives. Tracking metrics like tokens per request and total usage is essential for cost management, helping to optimize prompts and limit overuse.

• Bias and toxicity: LLMs learn from vast internet data and can inadvertently produce biased, offensive, or otherwise inappropriate content. They might, for example, exhibit gender or cultural biases in their responses or use toxic language. These ethical issues can harm users or a company’s reputation. Monitoring for bias and toxicity is crucial – for instance, using content filters or evaluators to flag outputs that contain hate speech or reflect unfair stereotypes. Observability tooling often includes safety checks or bias detection metrics to ensure the AI’s behavior aligns with ethical and inclusive standards.

• Security & privacy risks: In addition to prompt injection, LLM applications face other security concerns. The model might inadvertently leak sensitive data that was in its training set or provided context. Users might exploit the system to extract hidden prompts or confidential info. Also, if the application integrates with databases or external services, those components must be monitored for failures or breaches. Observability helps by logging all interactions and alerting on anomalies (e.g., a sudden spike in data volume or unusual access patterns). This ensures that any potential data leakage or unauthorized access can be quickly detected and addressed.

Many of these issues are interrelated – for example, a prompt injection could lead to a toxic or biased output (combining security and ethical problems), or a hallucination could go unnoticed if user feedback isn’t gathered. LLM observability directly targets these pain points by providing the visibility and tools needed to detect when they occur and understand their causes. Next, we will see how observability differs from standard monitoring in tackling such problems. 

LLM observability vs. LLM monitoring

 It’s important to clarify the distinction between LLM monitoring and LLM observability in the context of LLM applications. In traditional software operations, monitoring usually means keeping track of key metrics and system health indicators (CPU usage, error rates, throughput, etc.) and setting up alerts when things go out of bounds. Observability is a broader, deeper approach – it not only collects metrics but also logs, traces, and other data that allow you to explore and explain why something is happening in the system.For LLMs, the difference can be summarized as follows:

• LLM Monitoring focuses on the what. It continuously tracks performance metrics of the LLM application in real time to ensure the system is behaving as expected. This includes metrics like response latency, request throughput, API error rates, and usage counts (e.g. number of prompts or tokens processed). Monitoring can also involve simple checks on inputs and outputs – for instance, confirming the model is responding and maybe counting how often certain errors or keywords occur. The goal of monitoring is to detect operational issues quickly (like the service going down or responses slowing beyond a threshold) and trigger alerts or dashboards for engineers. It answers questions like, “Is the LLM service up and meeting our performance SLAs?”.

• LLM Observability goes beyond that to focus on the why. An observability solution provides full visibility into all the moving parts of the LLM system. It not only gathers the metrics that monitoring would, but also correlates them with logs, traces, and rich data about each request to enable debugging and in-depth analysis. With observability, engineers can reconstruct the path of a specific query through the system – see the prompt that was given, the steps the application took (e.g. calls to a vector database or multiple model invocations), and the output generated. This means when an issue is detected (say, a sudden spike in latency or a bizarre answer from the model), observability tools let you dig into the details and find the root cause: Was it a particular prompt that caused the model to loop? Was an external API call slow? Did a certain user input trigger a hidden bug? Tracing is a key component here: observability often involves capturing traces or timelines of each request across components, which is invaluable for diagnosing complex issues that involve multiple services or asynchronous steps. In short, monitoring might tell you something’s wrong, but observability helps you understand why it’s wrong and how to fix it.

To illustrate, imagine a scenario: your LLM-backed chatbot suddenly gives an incorrect (hallucinated) answer to a factual question, and also the response took 5 seconds instead of the usual 1 second. Basic monitoring might alert you that the error rate is up or that latency is high. But with observability, you could trace that specific request: the trace might reveal that the chatbot made a call to a knowledge base (vector database) which returned irrelevant data, causing the LLM to hallucinate an answer and take longer while searching.By examining the trace and logs, you discover the root cause – maybe a vector search query failed to retrieve the right document due to an indexing issue. This level of insight comes from having an observability framework in place, not just monitoring counters. LLM monitoring is necessary but not sufficient: it tells you the metrics status, whereas LLM observability gives you the power to ask arbitrary questions about your system’s behavior and get answers. In the next sections, we will dive into the unique challenges that make observability crucial for LLMs and what features an observability solution should have to address them.

Challenges in observing LLM systems

Building observability for LLM applications is hard because these systems present unique challenges beyond those of traditional software or simpler ML models. Here are some key challenges that LLM observability aims to address:

• Non-deterministic outputs: Unlike a classical algorithm that will produce the same output for a given input, LLMs are stochastic. The same prompt can yield different completions from one run to another. Moreover, LLM behavior can drift or change over time – for example, if you use a third-party API, the provider might update the model or training data, subtly altering its responses. This non-determinism means you can’t rely on fixed expected outputs or unit tests in the usual way. Small prompt wording changes or differences in input context can produce significantly different results. Traditional testing (comparing to a single “ground truth” answer) fails because there may be many acceptable answers, or the model may be unpredictably wrong in new ways. As Confident AI’s guide notes, “real users will always surprise you with something unexpected… LLM observability enables you to automatically detect these unpredictable queries and address any issues they may cause.” Continuous observability is needed to statistically detect when the model’s outputs start straying outside acceptable parameters, since you can’t pre-define all correct outcomes.

• Complex, multi-component pipelines: LLM applications often involve more than just a single model call. There may be retrieval components (e.g. searching a vector database for relevant context), various transformers or prompt templates, external API calls, and post-processing steps. Additionally, modern LLM systems might be composed of chained LLM calls or agents: for example, one LLM call’s output feeds into another, or an agent that iteratively decides to use tools or make more queries. This complexity makes it difficult to pinpoint where something went wrong. An error or slowdown could originate in the database lookup, the prompt formatting logic, the model inference step, or even in a downstream parsing of the model’s output. Debugging across these distributed components is a major challenge. Observability needs to tie together events from all parts of the pipeline. For instance, when a user question comes in, you want to trace it through the retrieval step (and see what documents were fetched from the vector DB), then through the LLM call (and capture the prompt and model response), and through any subsequent logic. Without this, you’re “left in the dark” about why a request failed or returned a bad answer. One common scenario: Did our knowledge base fail to return relevant data, or did the LLM mis-handle perfectly good data? Observability data (like logged retrieval results and model outputs) lets you do root cause analysis on such questions. In essence, an LLM observability solution must embrace distributed tracing and logging so that developers can see end-to-end flow across complex LLM pipelines.

• Black-box model reasoning: Even if we capture the flow of operations, the core LLM itself is often a black box in terms of how it arrives at a particular output. We don’t have explicit internal state or explainability from the model’s millions of neurons. This opacity can be frustrating: for example, why did the model output a biased statement? Why did it latch onto an irrelevant part of the prompt? Traditional ML observability might use techniques like feature importance or SHAP values for simpler models, but those don’t translate well to the giant opaque embeddings of an LLM. Explainability in LLMs is still an open research area, but observability tools try to provide proxies for it – such as showing the intermediate steps in a chain of thought, visualizing attention weights or embeddings, or correlating output changes with prompt changes. The challenge is to surface interpretable insights from the model’s behavior. While we can’t fully open the black box, observability can at least log things like which knowledge source was used, or provide semantic traces of how the model’s output evolves with each prompt refinement. For instance, techniques like comparing the embeddings of the model’s answer to known references can hint if the model was referencing the provided context or just relying on its internal memory. The non-deterministic and opaque reasoning of LLMs means observability has to capture rich context (prompts, embeddings, etc.) to give any hope of diagnosing issues like “the model often confuses similar concepts” or “the model is veering off-topic for certain inputs.”

• Infinite input possibilities: Users can ask literally anything from an LLM. This open-ended input space means you cannot anticipate every possible query or scenario in advance. There will always be edge cases and long-tail queries that reveal new failure modes. For example, a user might combine instructions or ask something in a format you never tested, causing the model to respond oddly. This is a challenge for observability because it must be proactive and adaptive. Rather than only focusing on known metrics or predefined test cases, LLM observability should help automatically surface when something unusual or problematic occurs in new inputs. One approach is to perform real-time evaluations on a sample of live traffic – essentially treating production as an ongoing test. If the system detects an anomaly (say, a sudden increase in user feedback thumbs-down on answers, or a spike in a “hallucination score” for outputs), that query can be flagged for analysis. Without observability, these rare but important cases might slip by until a user reports an issue. As one guide put it, you can test thoroughly before deployment, “but real users will always surprise you… the range of possible queries is endless, making it impossible to cover everything with complete confidence”. Thus, observability acts as a net to catch the unknown unknowns in production.

• Performance and scaling trade-offs: Another challenge is that LLMs are resource-intensive. Monitoring performance metrics isn’t just about keeping users happy; it’s about controlling resource usage and costs. An observability solution for LLMs needs to account for metrics like GPU/CPU utilization, memory usage, and token usage per request. It should help answer questions like: Can our infrastructure handle more concurrent users? or Which part of the pipeline is the slowest and needs optimization? For example, maybe the vector database lookup is adding 200ms overhead – is that the bottleneck or is it the model inference on large prompts? Observability should visualize such breakdowns. Scalability is a challenge: as the application scales, the observability system itself must handle lots of telemetry data without causing too much overhead or latency. Furthermore, the cost dimension means we want to observe not just technical metrics but also business metrics like cost per query. Non-deterministic token usage can make cost hard to predict, so continuous observability of token counts and their dollar conversion helps in optimizing deployments.

In summary, the challenges of LLM observability stem from the fact that LLM applications are unlike traditional apps – they are stochastic, complex, lack fixed expected outputs, and push the limits of current infrastructure. Observability practices are evolving to meet these challenges, providing the needed visibility to ensure such AI systems remain performant, reliable, and trustworthy.

Key features to look for in LLM observability tools

Given the above challenges, what capabilities should an ideal LLM observability solution provide? Whether you are evaluating commercial platforms or building your own toolkit, look for the following critical features that address the needs of LLM-based systems:

• End-to-end tracing of LLM chains: Modern LLM applications often consist of LLM chains or agent sequences, where one step’s output feeds into the next. It’s essential that your observability tool can trace and debug these chains step by step. This means capturing each sub-call or operation in the workflow – for example, the retrieval from a vector database, the assembly of the final prompt, the call to the LLM API, and any post-processing. By visualizing the complete request flow (as a trace with spans), you can quickly identify where things might be going wrong. For instance, if a chain is slower than expected, the trace will show whether the delay is in the LLM call or perhaps in an upstream data fetch. If an LLM agent gets stuck in a loop, tracing reveals which step is repeating. LLM chain debugging capability is thus a must-have: it turns the complex pipeline into a transparent sequence that engineers can inspect and optimize.

• Visibility into the full application stack: An LLM application doesn’t run in isolation – it sits on a stack that could include model servers (or API clients), databases (vector stores, caches), web services, and more. When a user sees a poor outcome, the root cause could be anywhere in this stack. Therefore, your observability solution should cover all layers of the LLM application, not just the model calls. This means correlating events from the infrastructure level (CPU/GPU usage, memory, network latency), the data layer (e.g. how long a database query took, did it return results), and the application logic (errors in code, slow functions in the chain). Full-stack visibility allows you to pinpoint, for example, that a response was slow not due to the model, but because a vector database was overloaded, or that outputs became inaccurate after a certain deployment of a prompt template or model version. Essentially, the observability tool should be able to tell you which component or layer is at fault when something breaks. This often involves integrating with existing APM (Application Performance Monitoring) data or cloud monitoring for components like databases, and linking those metrics to the LLM request traces.

• Explainability and anomaly detection: Since LLMs are black boxes, your observability platform should provide features to explain and evaluate model decisions as much as possible, and automatically flag unusual behavior. Explainability can come in forms such as: logging the input prompts and output responses (so you can inspect what the model saw and produced), visualizing intermediate representations (like embeddings or the retrieved context documents used for a query), or even showing attention weights or token probabilities if the model provides them. These help in understanding why the model responded a certain way or why it might have erred. On top of that, anomaly detection is crucial. The tool should be able to monitor the streams of inputs and outputs and detect outliers – for example, a sudden surge in negative user feedback, or the model starting to output a string of very long responses (which might indicate it’s rambling or looping), or an unusual drop in some quality score. Advanced observability solutions include out-of-the-box detectors for things like bias or toxicity in outputs, or significant deviations from normal usage patterns. For instance, if the model usually answers within 2 sentences and suddenly gives a 10-paragraph response, that could be flagged as an anomaly. By catching such anomalies automatically, the system can alert engineers to investigate potential issues (like a prompt change that caused the model to go off-track or a user attempting a prompt injection). In summary, look for tools that not only collect data, but also intelligently analyze it to surface quality issues, biases, or errors in real time.

• Integration and security features: An effective LLM observability solution must blend into your existing workflows and tech stack. This includes being able to integrate with the LLM providers or frameworks you use, as well as with your data stores and messaging systems. For example, if you use OpenAI’s API and a local vector database, the observability tool should have connectors or SDKs to instrument calls to both. Many tools support integration with popular libraries (like LangChain, Hugging Face, etc.) or open standards like OpenTelemetry for tracing. Scalability is also part of integration – the tool should handle your volume of data and allow you to aggregate or sample intelligently as needed. On the security side, because LLM applications often handle sensitive data, the observability platform should offer robust security and privacy capabilities. This includes PII redaction (automatically masking personally identifiable information in prompts or outputs that get logged) and protection against logging sensitive secrets inadvertently. It should also be secure in itself – with proper access controls, encryption, and compliance if needed (especially in enterprise settings). Some observability solutions even provide features to guard against prompt hacking by scanning inputs and outputs for known attack patterns. In essence, you want a tool that is enterprise-ready: it integrates smoothly with your environment (APIs, databases, devops tools) and maintains high security standards so that monitoring your model doesn’t become a vulnerability.

• Evaluation and feedback loops: While the above features cover monitoring and debugging, a strong LLM observability solution also supports continuous evaluation of model quality. This means having mechanisms to measure the correctness or relevance of outputs on an ongoing basis, not just when things go wrong. Some tools incorporate an evaluation framework where you can define metrics or tests (for example, using reference datasets or regex checks for certain answer formats) and have the tool automatically grade each response or periodically run evaluation jobs. Automated LLM evaluations could include checking factuality (perhaps by cross-verifying answers against a knowledge base), computing embedding similarities to expected answers, or applying NLP metrics like BLEU/ROUGE for summarization tasks. In addition, human-in-the-loop feedback capabilities are valuable: the tool might allow easy capture of user ratings (thumbs up/down) or reviewer comments on responses. Integrating these feedback signals into the observability dashboard lets you correlate objective metrics with actual user satisfaction. Over time, having this evaluation layer helps answer the question “Are our LLM outputs getting better or worse?” and identifies specific areas for improvement (e.g., maybe the model is good at general questions but consistently fails on a certain category of queries – continuous eval will catch that). While not every observability tool has built-in eval, the best ones do or at least integrate with external evaluation pipelines. The ability to replay scenarios is another related feature: you should be able to take a recorded prompt/response and try it against a new model version or prompt template to compare outcomes, facilitating A/B testing and prompt optimization in a safe way.

• User interface and filtering: Finally, an often underrated feature is a good UI/UX for analysis. Observability data is only as useful as your ability to derive insights from it. Look for solutions that provide interactive dashboards, search and filtering capabilities, and perhaps custom visualization builders. For example, if you have thousands of logged LLM interactions, you want to easily filter down to those that were flagged as failures or those from a specific user or those using a particular model version. Advanced filtering can allow slicing the data by metadata like conversation ID, prompt template version, or any custom tags you log. This way, if a problem is reported (“user X got a bad answer in session Y”), you can zero in on that session’s trace quickly. Or you can filter for all responses where your automated evaluation gave a low score, to see common patterns among those failures. A flexible querying interface (and the ability to export data for offline analysis) is highly useful for ML engineers to perform error analysis. Some observability platforms also let you define alerts on certain conditions – for instance, trigger an alert if hallucination rate goes above 5% or if average latency exceeds 2 seconds. This ties back into monitoring, but configured with the richer LLM-specific context. In summary, the observability solution should empower you to explore the data easily and get actionable insights, rather than drowning in logs.

To recap, LLM observability tools should provide comprehensive chain tracing, full-stack visibility, explainability and anomaly detection, easy integration with scale and security, and features for ongoing evaluation and debugging. Table 1 below summarizes some of the essential components and features that make up a robust LLM observability solution:

 Table 1: Key components and features of an LLM observability solution. Each component plays a role in ensuring you have insight into both the technical performance and the semantic output quality of your LLM application. 

Overview of the LLM observability tooling landscape

LLM observability is a new and rapidly evolving field, and a variety of tools and platforms have emerged to help teams implement the features discussed above. These range from specialized startups and open-source frameworks to extensions of traditional monitoring products. Without focusing on specific product names, we can categorize the types of LLM observability tools available and the common capabilities they offer:

• Open-source APM/tracing solutions with LLM integrations: Some open-source Application Performance Monitoring tools (for example, those built on OpenTelemetry) have begun adding support for LLM workloads. They excel at capturing traces, metrics, and logs in a unified platform. By instrumenting LLM calls (often via language-specific SDKs or middleware), they allow you to leverage standard distributed tracing for LLM chains. These solutions often require a bit of setup and expertise but can be very powerful and cost-effective, providing end-to-end visibility if you’re willing to integrate them. Common capabilities include custom dashboards for LLM metrics (like total LLM calls, token throughput, latency per model), and possibly pre-built instrumentation for popular LLM frameworks.

• Cloud monitoring & AIOps platforms: Many cloud-based monitoring services now advertise LLM monitoring features as part of their offerings. These are typically extensions to their existing logging/metrics systems to handle prompt/response data and LLM-specific alerts. They might provide easy integration if you’re already using the platform (e.g., an agent that hooks into your application to automatically capture LLM requests). Typical features are real-time metrics, anomaly detection (leveraging AI to spot unusual model outputs or usage patterns), and out-of-the-box alerts for things like “hallucination detected” or “prompt hijack attempt”. Some of these platforms can be quite robust and scalable, though they may come at enterprise pricing. They emphasize reliability, security, and integration with broader observability (so you can see your LLM stats alongside your microservice metrics, for instance).

• Specialized LLM observability & evaluation tools: A number of newer tools are purpose-built for ML/LLM observability and focus heavily on evaluation of model outputs and data quality. These tools often provide a suite of LLM-specific features such as: prompt versioning and experimentation, automated evaluation metrics (e.g., comparing model output to ground truth or checking for biases), dataset management for prompts and responses, and even model “comparison reports” when you try a new model or prompt variation. They may incorporate guardrails (business rules or AI-based filters to catch unwanted outputs) as part of the observability loop, effectively merging monitoring with active policy enforcement. The strength of specialized tools is that they deeply understand LLM use cases – for example, they might allow you to A/B test two models in production and observe which yields better responses, or trace not just the technical calls but the semantic changes in output quality. They often offer fine-grained filtering and search over your logged interactions, which is invaluable for analysis. The trade-off is that these might require you to adopt their framework for logging/evaluation, and there’s a learning curve to using the full feature set (especially if they introduce their own concepts for scoring or feedback).

• Experiment tracking & LLMOps platforms: There are also tools that originate from ML experiment tracking (for training models) which have extended into the LLMOps and observability space. These are tools data scientists might already use to track model training runs, and now they can be used to log prompt-response pairs, track model versions, and visualize metrics for LLMs. They are great during development and can also be adapted for production monitoring. Typically, they offer strong support for versioning (of models, prompts, datasets) and integrating with Jupyter notebooks or scripts. They might not have all the real-time alerting capabilities of a dedicated monitoring tool, but they excel at providing a workspace to analyze model behavior and share reports. For example, after deploying an update, you could log a bunch of interactions and then use the tool’s UI to compare the new model’s responses to the old one side-by-side, using built-in metrics. These platforms often have some support for visualizing embeddings, computing similarity metrics, or hooking in custom evaluation code. Many of them are extensible, so you can plug in your own checks (for fairness, toxicity, etc.) and log those as additional metrics.

Despite the variety, most LLM observability tools share a common set of capabilities aligned with the features we discussed earlier. They log prompts and outputs, track performance metrics (latency, token usage, errors), allow you to trace requests through chain/agent logic, and provide ways to assess output quality (either via automated metrics or manual review interfaces). They also tend to support integration with external data stores (like vector databases, for retrieving context) and will log the interactions with those stores for you (e.g., which documents were retrieved for a given query). In fact, observability for RAG (Retrieval-Augmented Generation) systems is a focus area: tools are adding features to monitor the retrieval step’s performance and relevance, because it’s so critical for factual accuracy. For instance, an observability dashboard might show the percentage of questions for which the retrieved context actually contained the correct answer – if that drops, you know your knowledge base or retriever might be at fault rather than the language model.It’s worth noting that because this field is fast-moving, new features are being added to tools frequently. As of 2025, many platforms are starting to incorporate AI-assisted analytics, such as using an LLM itself to summarize or explain trends in the observability data (e.g., “these cluster of outputs look like they might be hallucinations about topic X”). Tools are also improving at handling feedback loops – connecting user feedback directly into the observability reports and even enabling one-click fine-tuning or prompt adjustments when an issue is identified.In choosing an observability approach, consider factors like ease of integration (how quickly can you instrument your app?), scalability & cost (will it handle your traffic and how expensive is the data logging?), LLM-specific insights (does it have features beyond generic monitoring, like bias detection or chain visualization?), and team workflow fit (do your engineers and ML scientists find the UI and data useful?). Many teams start with a combination of open-source logging and custom scripts, then graduate to a more specialized platform as their usage grows. Regardless of the tool, implementing LLM observability is an iterative process – you’ll refine what you monitor and the alerts you set as you learn more about your application’s behavior in the wild. The encouraging news is that a range of frameworks and platforms now support LLM observability, and they continue to evolve as best practices emerge.Next, let’s put some of these ideas into practice. We will walk through a tutorial using W&B Weave, an open-source toolkit by Weights & Biases designed for LLM observability, to see how one can trace LLM calls, log outputs, and even detect issues like hallucinations in a simple application.

Tutorial: Tracking LLM outputs and anomalies with W&B Weave

In this section, we will demonstrate a simplified workflow for adding observability to an LLM application using W&B Weave. Weave is a toolkit that helps developers instrument and monitor LLM applications by capturing traces of function calls, logging model inputs/outputs, and evaluating those outputs with custom or built-in metrics.The scenario for our tutorial is a toy Q&A application: we have a function that calls an LLM to answer questions, and we want to track the behavior of this function over time, including detecting any hallucinated answers or biased content. We will use a synthetic setup (so you can follow along without needing a real API key, though Weave can integrate with real LLM APIs easily). The tutorial will cover:

1. Setup and instrumentation: Installing W&B Weave and annotating our LLM call function to enable tracing.
2. Logging LLM calls: Running the instrumented function to generate some example outputs and sending those logs to Weave.
3. Anomaly detection with guardrails: Using Weave’s built-in Guardrails scorers to automatically evaluate outputs for issues like hallucinations or toxicity.
4. Visualizing traces and metrics: Viewing the collected traces and metrics in the Weave UI to identify any anomalies.

   Let’s go step by step.

1. Install and initialize W&B Weave

First, install the weave library (it’s open-source and can be installed via pip) and log in to Weights & Biases with your API key (you can get a free key by creating a W&B account). In your environment, you would run:

pip install weave wandbwandb login # to authenticate with your W&B API key

For this tutorial, let’s assume we have done that. Now, in our Python code, we import weave and initialize a Weave project for our observability logs:

import weaveimport asynciofrom typing import Dict, Any# Initialize a Weave project (logs will go here)weave.init(“llm-observability-demo”)

Calling weave.init(“…”) sets up a project (replace with your chosen project name) and prepares Weave to start capturing data. You might be prompted to log in if not already authenticated. Once initialized, Weave will hook into any functions we decorate with @weave.op to trace their execution.

2. Instrument the LLM call with @weave.op

We have a function answer_question(prompt: str) -> str that uses an LLM to answer a given question. Normally, inside this function you would call your LLM API (e.g., OpenAI or a local model). For illustration, we will simulate an LLM response. We will also simulate that sometimes the “LLM” might hallucinate (we can do this by purposely returning an incorrect answer for certain inputs).Here’s how we instrument the function:

@weave.op() # This decorator tells Weave to track this function’s inputs/outputsdef answer_question(question: str) -> str: “””Call the LLM to get an answer to the question.””” # Simulate a response (for demo, we hard-code a couple of cases) if “capital of France” in question: return “The capital of France is Paris.” elif “president of France” in question: # Let’s simulate a hallucination here: return “The president of France is Napoleon Bonaparte.” # (Incorrect – hallucinated) else: # A default generic response return “I’m sorry, I don’t have the information on that.”

In a real scenario, inside answer_question you might use something like openai.Completion.create(…) or your model’s inference call. You would still decorate it with @weave.op() in the same way. The decorator ensures that whenever answer_question is called, Weave will log the call, including the input argument (question) and the returned result, along with execution time and any other metadata.We also included a docstring (which Weave can capture as well), and in our simulated logic we deliberately put a mistake: if asked “Who is the president of France?”, our fake LLM returns “Napoleon Bonaparte” – clearly a hallucination or error (Napoleon is long dead and not a president). This will help demonstrate how we catch such issues.Now, let’s call our function a few times to generate some data:

questions = [ “What is the capital of France?”, “Who is the president of France?”, “How many legs does a spider have?”]for q in questions: answer = answer_question(q) # calling the op (this gets logged by Weave) print(f”Q: {q}nA: {answer}n”)

When you run this loop, a few things happen:

• The answer_question function runs for each question.
• Because of @weave.op, Weave intercepts each call. It records the function name, the input (q string), and the output (answer string), as well as timing information.
• Weave streams this data to the W&B platform under the project “llm-observability-demo”. After each call, Weave typically prints a link in your console to view the trace in the browser.

  After running the above, your console might show something like: Each call to answer_question is considered a traced run, which you can inspect.

3. Adding guardrails for output evaluation

Now that we have basic logging, let’s introduce Weave’s Guardrails feature to evaluate the outputs. W&B Weave Guardrails provides pre-built scorers that automatically check LLM inputs or outputs for issues like toxicity, bias, or hallucination. We will use a hallucination detection scorer for our example.Weave’s scorers can be used in two modes:

• Guardrail mode: active prevention (e.g., refuse or modify an output if it’s flagged).
• Monitor mode: passive evaluation (just log a metric, but don’t change the output).

 For observability, we will use monitor mode to log whether our outputs are likely hallucinated. Under the hood, a hallucination scorer might compare the LLM’s answer with the provided context or use a separate model to fact-check the answer.Here’s how we might integrate a scorer in code (simplified):

# Create a custom scorer for hallucination detectionclass HallucinationScorer(weave.Scorer): @weave.op def score(self, output: str) -> dict: “””A simple hallucination detector that flags certain keywords.””” # A dummy check: flag as hallucination if certain keyword is in output. # In practice, this could call a model or check against a knowledge base. hallucinated = “Napoleon” in output # just for demo criteria return { “hallucination”: 1 if hallucinated else 0, “confidence”: 0.9 if hallucinated else 0.1 } # Create a dataset of our questions for evaluationdataset = [ {“question”: “What is the capital of France?”}, {“question”: “Who is the president of France?”}, {“question”: “How many legs does a spider have?”}] # Create an evaluationevaluation = weave.Evaluation( dataset=dataset, scorers=[HallucinationScorer()]) # Run the evaluationresults = evaluation.evaluate(answer_question)print(“Evaluation complete! Check the Weave UI for detailed results.”)

Let’s break that down:

• We define a HallucinationScorer class with a score method. This method inspects the output string and returns a dictionary with a metric (here “hallucination”: 1 or 0). In a real use case, you might implement a more sophisticated check, but for illustration we just flag if the output contains “Napoleon”.
• We use answer_question.call(…) instead of a direct call. The .call() method provided by Weave executes the function and also gives us a Call object representing that execution. We need this object so that when we log the scorer’s result, it attaches to the correct function call record in Weave’s database.
• Then we call the scorer’s score via .call() as well, passing in the output we got and the _call object of the original function call. This ensures the score (hallucination=1 or 0) is linked with that specific run.

  When the scorer runs, Weave will log the scorer’s output just like any other op. Since we’ve attached it to the original call, Weave knows this is an evaluation of that call. Now, in the Weave UI, every recorded call of answer_question will have an associated hallucination score. If our dummy logic is correct, the call for “Who is the president of France?” will have hallucination: 1 (meaning flagged) while the others have 0.We could similarly add other scorers – for example, a toxicity scorer to check if the output contains hate speech (not likely in our simple Q&A, but useful in general), or a relevancy scorer to check if the answer stayed on topic. Weave provides several built-in scorers that you can use out of the box, and you can easily create custom ones like we did above.Here’s an example with multiple scorers:

# Additional scorer for answer relevancyclass RelevancyScorer(weave.Scorer): @weave.op def score(self, question: str, output: str) -> dict: “””Check if the answer is relevant to the question.””” if “don’t have the information” in output: return {“relevancy”: 0.2} # Low relevancy for non-answers elif any(word in output.lower() for word in question.lower().split()): return {“relevancy”: 0.8} # High relevancy if answer contains question words else: return {“relevancy”: 0.5} # Medium relevancy otherwise # Update our evaluation with multiple scorersevaluation = weave.Evaluation( dataset=dataset, scorers=[HallucinationScorer(), RelevancyScorer()]) # Run the evaluation with multiple metricsresults = evaluation.evaluate(answer_question)

Which we can analyze in Weave: The key idea is that we are augmenting each LLM output with evaluation metrics. These metrics become part of our observability data – they can be visualized and aggregated. Weave’s evaluation framework essentially allows every LLM call to be automatically analyzed for quality and safety issues, and since all data is logged, we can later review how often hallucinations happened or if any outputs were flagged for bias. 

4. Visualizing traces and metrics in Weave

With data now logged (our prompt, response, and a hallucination flag), we head to the Weights & Biases interface to examine it. When you open the project in your browser, you will see a list of runs or traces. Each call to answer_question is recorded as an interactive trace.Key features you’ll find in the Weave UI:

1. Traces Tab: Shows all your function calls with inputs, outputs, and timing information
2. Evaluation Results: When you run evaluations, you’ll see aggregated results and can drill down into individual scored calls
3. Call Details: Click on any call to see full details including the evaluation scores attached to that specific call

  In the trace details, you can see:

• The function that was called (answer_question)
• The input (the question)
• The output (the answer)
• Timing information
• Any evaluation scores that were computed

    Crucially, because we ran an evaluation, the traces will show the evaluation results. For the “president of France” question, you would see something like:

• hallucination: 1 (flagged)
• confidence: 0.9
• relevancy: 0.8 (if using the relevancy scorer)

  While the other questions would show:

• hallucination: 0 (not flagged)
• confidence: 0.1

 Weave provides tools to filter and chart these metrics. You can create dashboards showing hallucination frequency over time, compare different model versions, and set up alerts if certain thresholds are exceeded.Advanced Analysis Features:

• Filtering and Searching: Filter traces by evaluation scores, time ranges, or specific inputs
• Comparison Views: Compare the performance of different models or prompt variations
• Aggregated Metrics: See summary statistics across all your evaluations
• Export Capabilities: Export your trace data for further analysis

Summary

This tutorial demonstrates a basic but complete observability workflow using W&B Weave:

1. Instrumentation: We decorated our LLM function with @weave.op() for automatic tracing
2. Logging: Every function call is automatically logged with inputs, outputs, and metadata
3. Evaluation: We created custom scorers to detect hallucinations and assess relevancy
4. Analysis: All data flows to the Weave UI where you can analyze patterns, set up alerts, and track model quality over time

   Key takeaways:

• Weave makes it easy to add observability to LLM applications with minimal code changes
• The evaluation framework allows you to systematically assess output quality
• All data is automatically linked and visualizable in the web UI
• You can scale this approach to production systems with real LLM APIs

   This is a basic example, but you can imagine scaling this up. In a production app with W&B Weave, you might trace entire user conversations (with each message as a trace span), log context retrieved from a vector database, and attach multiple evaluation checks (for factuality, bias, security). The result is a rich dataset that you can use to constantly improve your LLM system.Observability helps you iterate faster: you notice an issue (e.g., a pattern of mistakes), you fix something (change prompt or model), and then use the observability data to confirm the issue is resolved and no new ones have appeared.

LLM observability metrics and evaluation

Throughout this article, we’ve mentioned various metrics and evaluation criteria for LLM performance. Let’s bring those into focus and answer some key questions about how we assess LLM outputs and system performance. Broadly, there are two kinds of metrics to consider when we talk about LLM observability:

• Quality and behavior metrics – how good are the model’s outputs? Are they relevant, correct, fair, etc.? These help monitor the content the LLM generates.
• System and performance metrics – how well is the system running? How fast, how resource-intensive, how reliable? These ensure the operational health of the application.

 Both types are crucial to LLM observability, as they address the two fundamental aspects: making sure the model is saying the right things (quality) and doing so efficiently and safely (performance).

Quality evaluation metrics for LLM outputs

Measuring the quality of LLM outputs is challenging because it’s often subjective and context-dependent. However, there are several evaluation metrics and methods that can be used as proxies to quantify aspects of output quality. Here are some important ones:

• Relevance: This measures how well the model’s response addresses the user’s query or the task at hand. A relevant answer is on-topic and useful to the user’s request. In tasks like question-answering, relevance overlaps with correctness (a relevant answer is usually a correct one to the question). In dialogue, relevance means the response stays on topic. We can measure relevance via information retrieval metrics (does the answer contain the information found in reference texts for the query?) or via semantic similarity (comparing the embedding of the answer to that of an expected answer). Human evaluation is often used too (raters judge if an answer is relevant). Maintaining high relevance is important because an irrelevant or off-topic answer indicates the model misunderstood the query or veered off – something observability should catch. For example, if users ask “What is the weather tomorrow in Paris?” and the model gives a generic definition of Paris, that’s low relevance. Many observability setups include a relevancy scorer to automatically compare the question and answer embeddings – a low score might flag a potential issue.
• Perplexity: Perplexity is a classic metric from language modeling that measures how well a model predicts a sample of text. Technically, it is the exponentiated negative log-likelihood of the test data according to the model. In simple terms, perplexity gauges the model’s “surprise” or uncertainty when generating text – lower perplexity means the model finds the sequence more predictable (and thus it fits the model’s training distribution better). For LLM observability, perplexity is more often used as a model-level evaluation rather than per single output. It’s useful when assessing model versions or changes: if you have a holdout dataset, you can track perplexity to see if the model’s overall language understanding is improving. A lower perplexity on a relevant dataset generally correlates with a more fluent and potentially more accurate model. However, perplexity alone doesn’t guarantee task success – a model can have low perplexity but still hallucinate on factual questions, since perplexity doesn’t directly measure truthfulness. In monitoring, perplexity can be used to detect if a model starts performing oddly on the data distribution. For example, if suddenly your model’s perplexity on recent queries goes up, it might indicate distribution shift in inputs or that the model is struggling to generate likely text, often correlating with confusion or errors in output. Some advanced observability tools might calculate a form of prompt perplexity or use model-internal likelihoods to detect uncertain answers. In summary, perplexity contributes to assessing the model’s general confidence and can reveal drift or degradation in the model’s language performance.
• Fairness and bias metrics: Fairness in LLM outputs means the model’s responses are free from inappropriate bias or discrimination toward particular groups. Measuring this can be complex, as bias is context-dependent. Common approaches include bias benchmarks (sets of queries designed to elicit responses about different demographic groups to see if the model treats them differently) and statistical parity metrics (e.g., comparing sentiment of model outputs when the prompt only varies the gender or ethnicity in a scenario). For LLM observability, one might use metrics like the toxicity score or bias score provided by external classifiers. For instance, Jigsaw’s Perspective API can assign a toxicity probability to text; if your LLM outputs cross a certain toxicity threshold, you log that as an event. Similarly, one could define a simple fairness metric: feed the model a template like “Two surgeons are talking. One is a man and one is a woman. Who is better at surgery?” – if the model picks one gender consistently, that’s a bias sign. In live monitoring, direct bias evaluation might involve detecting protected attribute mentions in outputs and ensuring they’re not used in a derogatory or stereotypical manner. Some tools have bias detectors that look for certain phrases or unfair language. The contribution of fairness metrics is to ensure the model maintains ethical standards and a positive user experience. If an observability dashboard shows, for example, that responses containing ethnic terms have a higher toxicity score on average, that’s a red flag to address. Fairness metrics impact trust: by monitoring them, organizations can catch potentially harmful outputs before they escalate into public incidents. As the saying goes, “If you’re not measuring it, you’re missing it” – so integrating fairness checks into LLM observability is becoming a best practice for responsible AI deployment.

  Other quality metrics include accuracy (especially for QA tasks – whether the content is factually correct), coherence/fluency (does the text read well and logically?), and specificity (is the answer sufficiently specific vs. vague). Often, a composite of metrics or a custom scoring function is used. For example, you might define a scorecard for an answer that combines relevance, correctness, and clarity.Here’s a summary table of the key quality metrics discussed:

 Table 1: Example quality-focused metrics for evaluating LLM outputs. In practice, these metrics may be computed via automated tools or human annotation, and no single metric tells the whole story – but together they provide a multi-dimensional view of output quality. 

System performance and resource metrics for LLMs

Alongside output quality, observability must track operational metrics that reflect the performance and efficiency of the LLM application. These metrics are similar to those in traditional web services, but with some LLM-specific twists. Here are key observability metrics on the system side and their impact:

• Latency: Latency is the time it takes for the system to return a response to the user after receiving a request. In the context of LLMs, we often break this down further: for example, the time spent in the model inference vs. time spent in retrieval or other preprocessing. High latency can severely degrade user experience, especially in interactive applications where users expect quick answers. It’s important to monitor not just average latency but also tail latency (95th percentile, etc.), as occasional very slow responses can be just as problematic. Observability should link latency spikes to their cause – e.g., was a slow response due to a long prompt that took the model longer, or due to an external call (like a database search) hanging? By tracking latency per component, you can find bottlenecks. For instance, you might discover that prompt serialization is adding 100ms you could trim, or that enabling streaming could reduce perceived latency. Many teams set latency budgets (like, 1 second overall, with 0.7 sec for the LLM and 0.3 for all else) – observability data helps verify these are met and triggers alerts if breached.

• Throughput: Throughput measures how many requests (or tokens) the system can handle per unit time – essentially the capacity of the system. For LLM services, one might measure requests per second or, sometimes, tokens generated per second as a throughput measure. It’s particularly relevant for scaling: you want to know if your system can handle, say, 100 concurrent users or if it saturates at 20. Sometimes a model has great single-query latency but throughput suffers when parallel requests are high. Monitoring throughput alongside resource usage is key to planning scaling and load balancing. For example, if throughput plateaus and latency grows when you go beyond X qps (queries per second), that indicates you’ve hit a CPU/GPU or network bottleneck. Throughput impacts cost and user satisfaction – if your throughput is too low, requests will queue and users experience delays or timeouts. By observing throughput and maybe doing stress tests (and seeing those metrics in your observability dashboard), you can determine the safe limits of your system and set up autoscaling triggers appropriately. Throughput metrics also help in understanding efficiency: e.g., after an optimization, did your system handle more requests per second for the same hardware usage?
• Token usage & cost: In LLM applications, tracking token counts is extremely informative. Each request can be characterized by how many input tokens it had and how many output tokens were generated. This affects both latency and cost. If you see an unexpected increase in output token length over time, it might mean the model is rambling more or not following instructions to be concise. Monitoring token usage per request can thus signal changes in model behavior (maybe after a model update, answers got 20% longer on average – which might be good or bad). From a cost perspective, if you’re using a paid API, cost = (input tokens + output tokens) * price per token. Observability should include dashboards for total tokens used per hour/day and cost estimates, so you can catch if a particular user or feature is causing a spike. For example, maybe a poorly designed prompt caused the model to spew a 10,000-token response somewhere – you’d want to know that to avoid a surprise bill. Cost metrics tie in with business KPIs: you can watch cost per user request and work on prompt optimization to reduce it if needed. Additionally, token usage monitoring helps with rate limiting and quota management: if you have an internal policy to not exceed N tokens per day, you need to observe the token consumption trend.
• Resource utilization (CPU/GPU, memory): LLMs can be heavy on compute. If you are hosting the model yourself (on GPU or CPU), you need to monitor how utilized these resources are. High GPU utilization might mean you’re efficiently using hardware, but if it’s constantly at 100% and latency is suffering, you’re overloaded. Memory (RAM or GPU VRAM) is critical too – if memory usage is near capacity, you risk OOM errors. Observability should track these metrics ideally correlated with the LLM usage. For instance, it’s useful to see a timeline of GPU utilization alongside the number of requests in that interval, to identify saturation points. If you notice that memory usage grows over time (possible memory leak or accumulating cache), that’s something to fix. Disk I/O or network I/O could also be relevant if your LLM loads a lot of data or calls external services. The impact of resource metrics is mostly on system stability and scalability: they help ensure you’re not overcommitting your hardware, and they guide scaling decisions (e.g., add another GPU instance when CPU usage exceeds 70% consistently). They also help in performance tuning – e.g., if GPU isn’t fully utilized and CPU is idle, maybe you can increase batch size or concurrency to improve throughput. Conversely, if GPU is maxed out and CPU is low, the bottleneck is clearly the model computation.
• Error rates and failure modes: In any application, it’s important to monitor errors. For LLM apps, errors might include things like model service errors (e.g., an API call to the LLM provider fails or times out), invalid inputs (the user prompt couldn’t be handled), or downstream errors (e.g., unable to retrieve context, or your own code exceptions while formatting outputs). Observability should capture when a request does not complete normally. For example, if 2% of requests are returning errors (perhaps due to hitting a token limit or an external API outage), you need to see that trend and ideally get alerted. Content filtering can also be considered here: if the LLM refuses to answer certain queries (triggering its built-in safety filter), those could be logged as “no-answer” or filtered responses – essentially a kind of error from user perspective. Monitoring the frequency and triggers of those is useful (e.g., “we’re seeing many refusals for medical questions, maybe our prompt needs adjusting to allow them”). Error rate directly affects reliability – a spike in errors means users are not getting answers at all. It can also hint at bugs or external issues (if vector DB queries fail 5% of the time, maybe that service is flaky under load). Observability tools often include alerting on error rate thresholds because that’s a basic indicator of system health.

   To compile these into a visual summary:

 Table 2: Key performance and resource metrics for LLM applications. By tracking these, teams can ensure the system remains efficient and stable under real-world usage.With the combination of quality metrics (like relevance, correctness, fairness) and system metrics (like latency, throughput, usage), LLM observability provides a holistic view of an application’s health. You can tell not only “Is the system up and running within limits?” but also “Is it delivering the expected quality of output to users?”.

Conclusion

As we have explored, LLM observability is an essential discipline for anyone deploying large language model applications in the real world. It extends beyond traditional monitoring by giving us deep insights into both the performance and the behavior of our AI systems. By implementing observability for LLMs, teams can ensure their applications produce accurate, relevant, and safe outputs, while also running efficiently and reliably. This means we can catch hallucinations before they mislead users, mitigate prompt injection attacks or data leaks, reduce latency and cost issues, and maintain oversight of ethical considerations like bias and fairness. In a sense, observability is our compass for navigating the inherently unpredictable landscape of generative AI – it helps turn the black box of an LLM into a glass box, where we can see what’s going on and steer it in the right direction.In practice, building LLM observability involves capturing a rich set of telemetry: prompts, responses, intermediate steps, metrics, logs, and feedback. We then need tools to aggregate and analyze this data, from dashboards that plot key metrics to automated alerts and evaluations that flag anomalies. Thankfully, the tooling ecosystem is rising to the challenge. As we discussed, numerous frameworks (open-source and commercial) are available to help implement observability. Whether you use a specialized platform or assemble your own with open libraries, the important thing is to establish that feedback loop: observe -> diagnose -> improve -> repeat. For instance, using a tool like W&B Weave can jumpstart this process by instrumenting your application with minimal code changes and providing immediate visualization of how your model is performing. Our tutorial showed a glimpse of how easy it can be to log every LLM call and even attach evaluations for things like hallucinations or bias. With such setups, ML engineers and stakeholders can confidently track the quality of their LLMs over time and across versions.The benefits of incorporating LLM observability are clear. Teams gain faster debugging and iteration – instead of guessing why an AI behaved a certain way, they can pinpoint it in the logs or trace. They achieve better model performance and user satisfaction by continuously monitoring and refining prompts, models, and system parameters based on real data. Observability also provides a safety net for responsible AI, catching harmful outputs and enabling audits (critical for compliance and trust). Moreover, it helps with cost control and scaling, as you have detailed usage metrics to inform infrastructure decisions.As you develop and deploy LLM-powered applications, treat observability as a first-class concern, not an afterthought. Start with the question: “How will we know if our model is doing the right thing and doing it well?” and let that guide your observability design. By implementing the strategies and features outlined in this article – from tracing LLM chains to monitoring key metrics and using evaluation tools – you will empower your team to manage LLM applications proactively and effectively. The era of large language models is full of opportunities, but also uncertainties; with robust observability, we turn those uncertainties into actionable knowledge.If you’re ready to enhance your LLM application’s observability, consider exploring tools like W&B Weave. They can save you time and provide advanced capabilities out-of-the-box for tracing, evaluating, and improving your LLM. By investing in observability, you’re investing in the long-term success and scalability of your AI application.