Table of Contents

Observability architecture

This article explains how telemetry flows across the Neos ecosystem when OpenTelemetry is enabled.

For configuration steps, see OpenTelemetry collector configuration.

End-to-end flow

  1. Neos processes (gateway, report server, cluster backend, task runners, and other .NET services) produce traces, metrics, and logs.
  2. Processes send telemetry using OTLP to an OpenTelemetry Collector endpoint.
  3. The collector receives telemetry through OTLP (gRPC/HTTP) and optionally other receivers.
  4. The collector pipeline can filter, transform, enrich, and batch telemetry.
  5. The collector exports telemetry to one or more observability backends, such as Jaeger, Zipkin, Application Insights, or another external collector.
flowchart LR
    A[Neos .NET processes] -->|OTLP| B[OpenTelemetry Collector]
    B --> C[Receivers]
    C --> D[Processors<br/>filter transform enrich batch]
    D --> E[Exporters]
    E --> F[Jaeger]
    E --> G[Zipkin]
    E --> H[Application Insights]
    E --> I[External OTEL backend]

Collector deployment modes

The Helm chart uses observability.collector.mode to control collector deployment:

  • enabled: force internal collector deployment.
  • disabled: force external/direct routing (no internal collector deployment).
  • auto (default): deploy internal collector when legacy backends require it.

When the internal collector is active, all workloads target the internal collector URL. observability.otlp.global.endpoint applies only in direct OTLP mode (mode=disabled); it is ignored when the internal collector is running. Use per-signal endpoint overrides (observability.otlp.traces.endpoint, observability.otlp.metrics.endpoint, observability.otlp.logs.endpoint) to bypass the collector for individual signals.

Processing and governance

The collector is the control point for observability governance:

  • Filter noisy telemetry to reduce storage and ingestion costs.
  • Transform attributes to align naming conventions.
  • Enrich telemetry with environment, cluster, or tenant metadata.
  • Route selected signals to different backends.

Typical examples include removing low-value logs, normalizing service names, and adding deployment metadata.

Signal-specific notes

  • Traces and metrics are usually emitted through OpenTelemetry SDK integrations.
  • Logs can be emitted through OpenTelemetry-compatible logging sinks.
  • Inter-cluster traces are routed through Dapr sidecars using the Zipkin protocol. When the chart-generated collector configuration is active, Dapr is automatically routed to the collector's Zipkin receiver and observability.dapr.zipkinEndpoint is ignored. When a custom collector configuration is in use (config.inline / config.existingConfigMap), Zipkin routing is not guaranteed; observability.dapr.zipkinEndpoint must then be set explicitly when Jaeger or Application Insights is enabled. For full routing behavior, see OpenTelemetry collector configuration.

For log sink configuration with Serilog, see OpenTelemetry.

Operational recommendations

  • Keep defaults in one shared values section and override only what differs per environment.
  • Store sensitive OTLP headers or tokens in Kubernetes Secrets, not in plain values files.
  • Validate collector configuration changes before rollout in production.