The 3 Pillars of Observability

An overview of observability, its three pillars — logs, metrics, and traces — and why it matters in distributed systems.

Observability, in IT and cloud computing, refers to a set of processes and tools that let teams collect, aggregate, and correlate real-time data and events.

This makes it possible to analyze what's happening across a network, system, or environment — and to achieve better overall service outcomes.

It allows teams to better visualize and understand large computing networks and systems.

• Identify root causes of performance bottlenecks or issues
• Discover anomalous data patterns
• Help the business minimize downtime while maximizing reliability

Observability depends on system events to determine the what, when, and why.

The three pillars of observability are logs, metrics, and traces:

1. Logs: Archival records of system events and errors, in plain text or structured formats like JSON.
2. Metrics: Numerical measurements of system performance and behavior.
3. Traces: Representations of individual requests or transactions flowing through a system. Traces help identify dependency bottlenecks and root causes of issues.

Combining all three gives a holistic view into a system.

Logs are immutable records of discrete events occurring within a system.

Logs provide detailed information about the system, including:

• Event timestamps, transaction IDs, IP addresses, user IDs, the event or request itself, process details, and error messages.

Observability tools aggregate log data to help dev teams understand system failures and errors.

This can generate large volumes of data, requiring sophisticated log management tools.

Excessive logging can become noise.

Metrics are quantitative values that help analyze performance over time, including:

• KPIs
• Host Metrics: memory, disk and CPU usage
• Network Performance: uptime, latency, throughput
• App Metrics: response times, request and error rates
• Server Pool Metrics: # of instances

These are often summarized in time-series graphs to help assess health and trends.

Metric thresholds tied to alerts can help staff stay on top of current and impending issues.

Logs and metrics help teams understand individual system behavior. Traces, by contrast, give visibility into the lifetime and journey of requests through a system.

• End-to-end journey of a request through a system or network
• Captures the path and lifespan of each component involved in processing the request
• Especially useful for large, distributed systems
• Requires instrumentation across systems (coordination)

• https://www.ibm.com/think/insights/observability-pillars
• https://www.crowdstrike.com/en-us/cybersecurity-101/observability/three-pillars-of-observability/
• https://www.youtube.com/watch?v=q-T_NmS2Q_w