I’ve been using Observium for a very long time. Looking through this
site recently, I realised that quite a few of my old Observium articles
are still online, but I’ve never really written down how I became
involved with the project in the first place.

It goes back to my time at eBay and eBay Classifieds Group, where I was
working on the network infrastructure behind businesses including
Marktplaats.nl.

At that scale, monitoring obviously wasn’t optional. We had a large
amount of network infrastructure and needed to know not only whether
devices were reachable, but what was actually happening inside the
network: interfaces, BGP sessions, processors, memory, storage, sensors
and all the other things that can tell you that something is going wrong
before it becomes visible to users.

I came across Observium, which at the time was still a relatively
young project being developed primarily by Adam Armstrong.

I got to know Adam quite well as we started using Observium more
seriously and became more involved with its development, and over the
years I’ve come to consider him a personal friend. At the time, though,
the reason I became interested in Observium was simply that I liked its
approach to network monitoring.

It was very good at automatically discovering what was actually present
on a device. Instead of spending huge amounts of time manually defining
every interface, sensor and BGP session, Observium could discover the
network and build a useful picture of it automatically.

We started using it, and I became increasingly involved with the
project.

One of the things that was missing at the time was a proper alerting
system.

Observium was already very good at showing you the state of a network,
but in a production environment we also needed it to actively tell us
when that state changed in a way that required attention. Monitoring
something and periodically looking at a graph is one thing; knowing
immediately that a BGP peer has gone down or that an important resource
has crossed a threshold is another.

Through eBay/Marktplaats, I sponsored development of Observium and
specifically had the original alerting functionality developed.

The idea was to build alerting around the entities Observium was already
discovering. Rather than maintaining a completely separate set of
monitoring definitions, alerts could operate on things Observium already
understood: devices, ports, BGP peers, processors, memory, storage and
sensors.

That eventually became the entity-based alerting system.

Once it existed, the next problem was actually configuring it properly.

The alerting system was flexible, but that also meant you needed to
understand its metrics, attributes, conditions and association rules.
There weren’t many ready-made examples available at the time, so in 2014
I wrote “Useful Observium alert checker examples” on this site.

That article contained examples for the things I thought most network
operators would actually want to monitor: devices going down, CPU and
memory utilisation, BGP sessions, interface errors and discards, storage
utilisation, traffic levels and sensors.

Looking at it again more than a decade later is quite interesting
because the terminology and implementation have evolved, but much of the
basic alerting model is still recognisable.

Alerting continued to develop after that.

One limitation of the original system was that it was poller based. If
the poller ran every five minutes, there were some events that you
either wouldn’t see immediately or couldn’t sensibly detect through
polling at all.

That led to another feature I was interested in: syslog alerting.

When that became available in Observium, I wrote another article
explaining how to use it. Syslog alerting made it possible to react in
real time to messages generated by network devices—for example OSPF
events, duplicate IP or MAC-address warnings and other conditions that
don’t fit naturally into normal SNMP polling.

Over the years I wrote several other Observium-related articles and bits
of documentation, and maintained some of the material as the software
changed.

I wasn’t one of Observium’s developers, and I don’t want to rewrite
history to suggest that I was. Adam and the other Observium developers
built the software. My involvement was from the other side: I was
running networks, using the software in a real production environment,
sponsoring development of functionality we needed and helping document
how some of it could actually be used.

The alerting system was probably the most substantial part of that
involvement.

Observium has developed enormously since then. The current system has
alert checkers, notification transports, scheduled maintenance, an API,
syslog alerting and considerably more functionality than the version we
were working with at the time.

It’s nice to see that something we needed for our network more than a
decade ago became part of a monitoring system that is still actively
developed and used today.

And it’s also why there are so many old Observium posts scattered around
this website.

I wasn’t writing them as blog content.

I was writing down how we got things to work.