Your AI application may be running inside your own cloud account, but that doesn’t necessarily mean your observability data is.
A trace can contain much more than latency and token counts. Depending on how the application is instrumented, it may include prompts, model responses, retrieved documents, tool calls, user identifiers, metadata and other details from the workflow. Sending that data to an observability platform therefore becomes a deployment and data-governance decision, not just a tooling decision.
Observability is only one part of the wider AI stack, sitting alongside the infrastructure, model, data and application layers that support a production AI system. Our guide to the AI stack breaks down those layers and shows where observability fits into the broader architecture.
That’s why self hosted LLM observability and SaaS LLM observability need to be compared differently. The useful questions aren’t simply which dashboard looks better or which platform has more features. Where does the telemetry go? Who controls the infrastructure? Can the data stay within a required region or network boundary?
For teams with strict data residency or privacy requirements, those questions can narrow the shortlist before feature comparison even begins.
Table of Contents
Why Deployment Model Decides the Shortlist
The deployment model tells you who is responsible for the platform and, more importantly, where your observability data is processed and stored.
With SaaS, the vendor runs the infrastructure, handles upgrades and takes care of much of the operational work. That’s convenient, particularly for teams that don’t want another production system to maintain. The trade-off is that telemetry moves into the vendor’s environment, subject to its hosting regions, security controls and data-handling policies.
Self-hosting reverses that arrangement. Your team runs the observability platform on infrastructure it controls. That can mean a cloud environment such as AWS, Azure or GCP, or an on-premises environment. It gives you more control over data location and network access, but the responsibility for upgrades, scaling, backups and infrastructure security also comes with it.
Then there’s hybrid or BYOC deployment. Here, the vendor and customer split responsibility in some way, allowing parts of the platform to remain managed while data or workloads stay within the customer’s environment.
None of these models is automatically the “secure” choice. SaaS reduces operational burden. Self-hosting gives more infrastructure and data control. The right option depends on what the organization is actually required to control. If you’re also evaluating observability beyond LLM applications, our guide to the best AIOps platforms for IT teams looks at how broader infrastructure and operations platforms handle telemetry and monitoring.
That distinction matters when evaluating on premise LLM monitoring. “Not SaaS” doesn’t necessarily mean everything has to sit in a physical server room.
Self-Hosted vs SaaS LLM Observability
| Consideration | Self-hosted | SaaS |
|---|---|---|
| Infrastructure | Customer-managed | Vendor-managed |
| Data location | Customer-controlled | Vendor infrastructure |
| Updates | Customer responsibility | Vendor-managed |
| Data residency | More control | Depends on vendor regions |
| Network boundary | Can remain private | Requires vendor connectivity |
| Scaling | Customer responsibility | Vendor-managed |
| Operational burden | Higher | Lower |
The important detail is that self-hosting doesn’t automatically mean on-premises.
A company can deploy an observability platform inside its own AWS, Azure or GCP environment and still have a cloud-based architecture. If the requirement is simply to keep telemetry inside a particular VPC, that can be very different from a requirement for a fully on-premises or air-gapped deployment.
The same distinction applies to SaaS. A vendor may offer regional hosting, but that isn’t the same thing as giving the customer control over the underlying infrastructure.
VPC, On-Prem, and BYOC Support by Vendor
For Langfuse, the current self-hosting documentation supports deployment locally, on cloud infrastructure or on-premises. Production-scale deployments include Kubernetes and cloud deployment options for AWS, Azure and GCP. Langfuse also describes deployment within a VPC, with internet access being optional depending on the setup.
LangSmith takes a different route for organizations that need to run the platform themselves. Its self-hosted offering is positioned for Enterprise customers, allowing the platform to be hosted in the organization’s own infrastructure.
That makes the deployment question more useful than a simple “does it support self-hosting?” checkbox. A procurement team should ask where the platform can run, which components remain under its control, and what network boundary the deployment can actually maintain.
What Data Does LLM Observability Actually Store?
An observability platform may receive prompts and completions, but that’s only part of the picture. Depending on the instrumentation, traces can also include retrieved context, tool arguments and outputs, session or user identifiers, model information, token usage, latency and errors.
That changes the privacy conversation.
A trace isn’t always just a technical log. For a customer-support agent, for example, the trace could contain the customer’s original message, information retrieved from an internal knowledge base and the response generated by the model. This becomes even more important when an application uses agents that retrieve information, call tools and move data across multiple steps. Our practical guide to building AI agents for scalable business operations covers these workflows, including the architecture and controls that become important as agents move into production.
That concern is also reflected in OWASP’s guidance on sensitive information disclosure in LLM applications, which covers risks involving PII, financial information, health records and confidential business data.
This is why LLM observability data residency and LLM data privacy need to be considered together. Before choosing a platform, a team should know exactly what it sends, where that information is stored and which controls are available to limit what gets captured.
SOC 2, ISO 27001 and ISO 42001 Status
Compliance badges are useful, but they’re not a substitute for checking the actual scope of a service.
Langfuse Cloud currently lists SOC 2 Type II and ISO 27001 certifications, along with GDPR compliance and a HIPAA-ready region. LangSmith also states that its hosted service is SOC 2 Type II, HIPAA and GDPR compliant, with additional enterprise controls and self-hosted deployment options.
ISO 42001 needs more care. I wouldn’t mark either vendor as ISO 42001 certified based on the current sources reviewed here. If that certification is a procurement requirement, it should be verified directly rather than inferred from other security certifications.
| Vendor | SOC 2 Type II | ISO 27001 | ISO 42001 | Relevant deployment |
|---|---|---|---|---|
| Langfuse | Yes | Yes | Not verified | Cloud and self-hosted |
| LangSmith | Yes | Not verified | Not verified | Cloud and Enterprise self-hosted |
There is another distinction worth keeping in mind: a vendor’s certification describes the controls covered by its audited service or organization. It doesn’t automatically make a customer’s own deployment compliant. Your configuration, access controls, retention policy and handling of sensitive data still matter.
EU Data Residency: What Does “EU Hosted” Actually Mean?
“EU hosted” sounds straightforward until you ask what is actually included.
Langfuse currently has a dedicated EU Cloud region in Ireland, and its documentation says the regions are separated from one another. It also documents where its databases, application infrastructure and backups are located.
LangSmith states that its hosted EU environment stores data in the Netherlands, while Enterprise customers can run LangSmith on their own Kubernetes cluster in AWS, GCP or Azure so the data stays within their environment.
For a procurement review, don’t stop at the region selector. Check where telemetry and backups are stored, whether support systems or subprocessors receive data, and whether data can cross the selected jurisdiction.
That’s what makes data residency a real control rather than a checkbox.
PII Redaction and Sensitive Data Controls
The safest place to remove sensitive information is before it reaches the observability platform.
Client-side masking can prevent selected data from being sent in the first place. Server-side masking happens after the observability service receives the data, so it solves a different problem.
Langfuse supports client-side filtering and also documents server-side data masking as an Enterprise feature for self-hosted deployments. Its OpenTelemetry integration can also filter spans before they are exported.
That distinction is easy to overlook. If a policy says a particular PII value must never leave the application environment, masking it after ingestion isn’t enough.
Retention Controls and Deletion
Retention is another part of the data-security picture that gets less attention than encryption.
Keeping traces for six months may be useful for debugging and evaluation. Keeping the same customer data indefinitely probably isn’t.
Langfuse Cloud documents data retention and deletion controls, while its self-hosted Enterprise Edition includes configurable data retention policies. Self-hosting also gives the organization control over the underlying storage, but that means the organization becomes responsible for its own retention, backups and deletion policies.
So when comparing platforms, ask two separate questions: How long does the vendor retain my data by default, and how much control do I have over changing that period?
Licence Restrictions on Self-Hosting
This is where “open source” and “self-hosted” can get confused.
Langfuse’s core self-hosted offering is MIT licensed, and its documentation says the core features and APIs are available in OSS without limits. Some additional capabilities require an Enterprise license, including project-level RBAC, protected prompt labels, retention policies, audit logs, server-side masking, SCIM and certain administration APIs.
LangSmith approaches self-hosting differently. Its own documentation positions self-hosted LangSmith as an Enterprise offering rather than a freely available self-hosted version.
That difference matters in procurement. “Can we run it ourselves?” isn’t enough. The better question is:
Which security and governance features are included when we run it ourselves?
Self-Hosted LLM Observability vs SaaS: Compliance Matrix
| Tool | Self-hosted | VPC / private environment | On-prem | EU hosting | PII controls | Retention controls | SOC 2 Type II | Enterprise self-hosting |
|---|---|---|---|---|---|---|---|---|
| Langfuse | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Optional |
| LangSmith | Yes | Enterprise | Enterprise options | Yes | Enterprise controls | Plan/deployment dependent | Yes | Yes |
How to Verify Claims Before Buying
Security pages tend to use broad phrases such as “enterprise-grade security” or “compliance-ready.” Those phrases aren’t very useful on their own.
Ask for the details behind them.
Start with the deployment architecture. Where does the application run, where are traces stored, and which services sit outside your environment? A data flow diagram can answer questions that a product page can’t.
Then check the data residency documentation, subprocessor list, retention and deletion policies, encryption details, access controls and SOC 2 evidence. If self-hosting is involved, look at the license too. Some platforms make the core software available for self-hosting while putting specific governance features behind an Enterprise license.
The goal isn’t to collect security badges. It’s to understand what actually happens to your data after the SDK sends a trace.
When Self-Hosted LLM Observability Makes Sense
Self-hosting starts to make more sense when the observability data itself is subject to strict requirements.
That could be customer information inside prompts, regulated data that can’t leave a particular region, or an internal security policy that requires telemetry to remain inside a private network. It can also be the practical choice when an organization already operates its own Kubernetes or cloud infrastructure and wants observability to stay within that environment.
There is another case that’s easy to overlook: retention.
If the security team needs direct control over how traces are stored, backed up and deleted, self-hosting provides more room to build those policies around existing infrastructure.
But self-hosting isn’t automatically safer. You also become responsible for securing the platform, maintaining its databases, managing upgrades and making sure backups don’t quietly become another copy of the same sensitive data.
When SaaS LLM Observability Is the Better Choice
For some teams, running another production platform simply isn’t worth the additional work.
SaaS can be a better fit when the team wants to start quickly, doesn’t have a requirement for infrastructure ownership and is comfortable with the vendor’s security, residency and retention policies.
The managed model also removes much of the operational work. You don’t have to maintain ClickHouse, PostgreSQL, Redis or the surrounding infrastructure yourself, and upgrades are handled by the provider.
That doesn’t mean you skip the security review. It means the review focuses more heavily on the vendor’s data handling, hosting regions, subprocessors, access controls and contractual commitments.
If those controls meet your requirements, paying for a managed service may be more sensible than building and maintaining the same stack internally.
The Procurement Checklist
Before choosing a platform, confirm:
- Where are traces stored?
- Can prompts and responses be masked before ingestion?
- Can data remain in a specific region?
- Is on-premises or VPC deployment supported?
- What does the self-hosted license actually include?
- Which security features require Enterprise?
- What are the retention and deletion controls?
- What compliance evidence can the vendor provide?
- Who operates the infrastructure?
- Where are backups stored?
These questions turn a broad security review into something a procurement or engineering team can actually work with.
FAQs
Which LLM observability tools can be self-hosted?
Several LLM observability platforms offer self-hosted deployments, but the terms differ. Some provide an open-source core, while others restrict self-hosting to Enterprise customers. Check the license, deployment requirements and feature restrictions rather than treating every “self-hosted” label as equivalent.
Which LLM observability tools are SOC 2 Type II certified?
SOC 2 Type II status varies by vendor and service. More importantly, check what the report covers and whether it applies to the specific hosted or self-hosted deployment you’re considering. A vendor’s certification doesn’t automatically make your own configuration compliant.
Which LLM observability tools keep data in the EU?
Some platforms provide dedicated EU hosting regions, while others offer private or self-hosted deployments that let customers control the infrastructure location. Check more than the primary database. Storage, backups, subprocessors and data-transfer policies can affect where information actually goes.
Do I need PCI compliance for LLM observability?
It depends on whether payment-card data can enter your prompts, responses or traces. If your application handles cardholder data, the observability layer needs to be included in the broader PCI DSS assessment. Preventing that data from entering traces in the first place can reduce the scope.
Do I need HIPAA compliance before storing LLM traces?
If traces can contain protected health information, HIPAA requirements may become relevant. The important questions are whether the platform supports the required safeguards, whether the appropriate contractual arrangements are available and whether your configuration prevents unnecessary PHI from being captured.
Does self-hosting guarantee that LLM data stays private?
No. Self-hosting gives you more control over where the platform runs and where its data is stored, but privacy still depends on configuration. Network access, encryption, permissions, logging, backups and third-party services all matter.
Is self-hosted LLM observability cheaper than SaaS?
Not necessarily. A self-hosted license may be free or lower-cost, but infrastructure, storage, backups, engineering time, maintenance and scaling still have a cost. The comparison should be based on total operating cost rather than the software license alone.
Does self-hosting mean on-premises?
No. Self-hosting means your organization runs the platform. That can happen on physical on-premises infrastructure or inside a cloud environment such as AWS, Azure or GCP. An organization can therefore have a self-hosted deployment without running its own data center.
Final Verdict
The right LLM observability deployment isn’t the one with the longest security checklist. It’s the one whose data flow, deployment model, compliance controls and licensing terms fit the way your organization actually operates.
Self-hosting gives you more control, but also more responsibility. SaaS removes much of that operational work, but requires you to examine the vendor’s data residency, retention, subprocessors and access policies closely.
For procurement teams, that trade-off matters more than the feature count.
Leave a Reply