IODA Signal
IODA (Internet Outage Detection and
Analysis) monitors Internet outages from several independent data sources. Its
BGP signal is the number of /24 blocks an AS, country or region has visible in
the global routing system, computed every five minutes from all Route Views
and RIPE RIS collectors. quickrib.observers.ioda reproduces that signal for
ASes, and the alert rule IODA applies to it. The method is described on the
IODA help page.
Definition
Let \(F\) be the set of full-feed peers and \(n(p)\) the number of them that have a route to prefix \(p\). The prefix is visible when a majority of the full-feed peers see it:
The signal of an AS is the number of /24 blocks covered by the union of the visible prefixes it originates. The union matters: a more specific nested in a visible less specific adds no blocks, and a prefix longer than a /24 adds none either, since it does not fill a block. When several ASes originate the same prefix, each is credited with it.
IODA defines a full-feed peer by an absolute table size (more than 400k IPv4 or 10k IPv6 prefixes). QuickRIB uses its own relative rule, more than 80% of the collector's distinct prefixes, and the observer takes the selection the pipeline already made. The two rules select different peers at a dual-stack collector; see Reconstruction.
An AS is alerting in a bin when its value falls below 99% of the median of the previous 24 hours. Consecutive alerting bins form an outage event, scored as
A bin that is itself alerting is excluded from the median, otherwise a long outage would lower its own baseline and declare itself over while still down. Neither the exclusion nor the score formula is documented by the API; both were recovered by matching its alert and event records.
Usage
"""Reproduce IODA's BGP signal and detect an outage in it.
The window brackets a real outage: AS43160 loses roughly a quarter of its
visible /24s at 04:20 UTC and recovers at 05:45.
`IODAObserver` needs no reference to the RIB. The pipeline hands it the
full-feed selection before the replay starts, so it accumulates straight out of
`update_rib`. Running with `fullfeed_only=True` (the default) is therefore both
correct and roughly twice as fast, because the peers it would ignore never enter
the table.
"""
import datetime
from pathlib import Path
from quickrib import QuickRIB, QuickRIBConfig
from quickrib.observers import IODAObserver, detect_events
OUTAGE_ASN = "43160"
config = QuickRIBConfig(
start_time=datetime.datetime(2022, 1, 26, 4, 0),
end_time=datetime.datetime(2022, 1, 26, 6, 30),
collectors=["route-views.eqix"],
dump_res=datetime.timedelta(minutes=5), # IODA's native step
cache_dir=Path("cache"),
parser="bgpkit",
)
quickrib = QuickRIB.from_config(config)
observer = IODAObserver(entities=[OUTAGE_ASN])
quickrib.rib.attach_observer(observer)
quickrib.run()
# observer.series maps a 5-minute bin to {origin ASN: visible /24s}.
series = {bin_ts: totals[OUTAGE_ASN] for bin_ts, totals in observer.series.items()}
for bin_ts, visible in sorted(series.items()):
when = datetime.datetime.fromtimestamp(bin_ts, tz=datetime.UTC)
print(f"{when:%H:%M} {visible:>4} visible /24s")
# The alert rule needs 24 hours of history to establish a baseline, so a short
# replay cannot trigger it. Feed it a longer series to get IODA-shaped events.
for event in detect_events(series, location=f"AS{OUTAGE_ASN}"):
print(f"outage at {event['start']} lasting {event['duration']}s, "
f"score {event['score']:.0f}")
observer.series maps the start of each five-minute bin (a Unix timestamp) to
{origin ASN: visible /24s}. Pass entities=[...] to restrict the output to a
few ASes, and ip_version=6 for /48 blocks, in which case an explicit
ff_peers set is needed at a dual-stack collector.
IODAObserver accumulates its counts while the table is built and keeps them
current in the update path; a dump only recounts the ASes whose visible set
changed. RibIODAObserver recomputes the signal from the whole table at every
dump and is the reference the incremental observer is tested against. They
agree exactly.
evaluate_series, detect_alerts and detect_events apply the alert rule to
any series, including one fetched from the API. The rule needs 24 hours of
history, so a short replay alone cannot trigger it.
Accuracy
IODA computes the quorum over every collector; a QuickRIB run reads a few, so the quorum is taken over a smaller peer population and the two signals can only be close. On the pinned test window, a single Route Views collector lands within 0.35% mean relative error of the API across 28 ASes and reproduces an outage on the API's own values, with the drop one bin late at most. Two effects remain:
- QuickRIB reports the table at a bin boundary while IODA's bin covers the interval after it, so a transition lands up to one bin late.
- Routes return to one collector over several minutes, while IODA sees whichever of its collectors relearns them first, so a recovery can be a few bins late.