Virtual Rejekts 2020

Watching the Watcher: Advanced Integrated Monitoring with Prometheus Federation
04-01, 18:30–19:00 (UTC), Track 1 (UTC)

The name Prometheus is no longer synonymous with "The Messenger of the Gods". It has far greater responsibility in today's cloud native application environments.

This session will cover configuring Prometheus in a production environment with a focus on federating Prometheus deployments together.

Federation is the term that describes using one instance of Prometheus to scrape metrics from another instance, and it's not often talked about. Teams deploying Prometheus may want to use federation for any number of reasons:
- Their existing Prometheus installation is projected to outgrow its current hardware
- To manage performance as the number of samples collected by Prometheus grows
- Integrate a Prometheus deployment from a different application

Each of these topics will be explained and an example of federation will be demonstrated using Linkerd which includes its own Prometheus.


This talk is meant for individuals who are exploring new ways to use cloud native technologies and want to know more about components which are becoming ubiquitous, like Prometheus.

Attendees will learn what Prometheus offers as a time-series data store, as well as what it does not offer.

After the basics of Prometheus are addressed, the talk will go into the anatomy of a Prometheus installation covering architecture and hardware requirements. Next, the talk will cover Prometheus as a component in a distributed application in a production environment, where there may be other instances of Prometheus running.

As a result, attendees will learn how to install and configure a standalone Prometheus as part of a larger distributed application with basic security and redundancy so that it can recover quickly in the event of a catastrophic outage.

Next, attendees will learn how to link (federate) the standalone instance of Prometheus with the instance of Prometheus that is deployed with the Linkerd service mesh.

Finally, the session will discuss alternatives to Prometheus federation using the CNCF project Cortex.

Charles Pretzer is a field engineer at Buoyant, where he spends his time collaborating and engaging with the open source community of the CNCF service mesh, Linkerd. He has spent time using Prometheus to monitor and debug microservices applications and understands the value that high-cardinality metrics provide. Charles has spoken at meetups and conferences hosted by ABN Amro, Macnica, and NGINX Conf. When he's not presenting or in front of a computer, he's riding a motorcycle or making a delicious mess in the kitchen.