Prometheus Pitfalls and the Rise of Continuous Profiling - OpenObservability Talks S2E07

OpenObservability Talks

We’ve grown to rely on “the three pillars” for observability - logs, metrics and traces. Popular frameworks such as Prometheus have helped popularize these practices. But now people are starting to realize that it’s not enough.

On this episode Dotan Horovits will host Frederic Branczyk for a discussion about the unspoken pitfalls of Prometheus and the challenges of current observability coverage. We will also discuss the rise of Continuous Profiling as a new observability signal, what it’s about and where it can help. We’ll also review the recent launch of Parca, an open source project for continuous profiling that traces its roots to Red Hat’s internal ConProf open source tool.

Frederic is the founder and CEO of Polar Signals. Before founding Polar Signals he was a senior principal engineer and the main architect for all things Observability at Red Hat, which he joined through the CoreOS acquisition. Frederic is a Prometheus and Thanos maintainer as well as the tech lead for the special interest group for instrumentation in Kubernetes. In a previous life, he was a security researcher working on key management solutions as well as intrusion detection systems. When not working on software Frederic enjoys obsessing over brewing a perfect cup of coffee.

The episode was live-streamed on 16 December 2021 and the video is available at https://www.youtube.com/watch?v=G02g63oI0IA

OpenObservability Talks episodes are released monthly, on the last Thursday of each month. The episodes are also live-streamed on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.

Show Notes:

• The limitations of the three pillars model of observability
• Prometheus strengths and pitfalls
• how to start with continuous profiling
• how to correlate between different telemetry
• Parca OSS intro
• eBPF turned out perfect for instrumenting continuous profiling
• Parca OSS future plan
• how is the performance penalty of continuous profiling kept low
• what's the solution for high cardinality in Prometheus?
• will Parca OSS be contributed to an established OSS foundation?
• Prometheus Agent mode released
• OTEL operator now has an instrumentation CR
• continuous profiling support for interpreted languages

Resources:

• https://www.parca.dev/
• https://github.com/google/pprof
• https://increment.com/containers/observing-containers-pillars-of-observability/
• https://ebpf.io/
• https://research.google/pubs/pub36575/

Social:

• Twitter: https://twitter.com/OpenObserv
• YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠