Privacy adversaries may use BGP hijacking attacks to gain access to a bigger portion of Tor traffic than they would be able to see otherwise. BGP hijacking can be used to reroute internet traffic (for traffic interception and correlation attacks) or to perform denial of service against specific relays to increase the chance that Tor users select attacker-controlled relays.
The RAPTOR and Counter-RAPTOR papers by Sun et al. studied the effect of BGP attacks on Tor, in their paper they note:
more than 90% of BGP prefixes hosting Tor relays have
prefix length shorter than /24, making them vulnerable to
more-specific prefix attacks
We aim to measure the BGP routing security properties of prefixes that make up the Tor network. We put prefix properties in context with their Tor capacity (consensus weight fraction). The consensus weight fraction takes relay bandwidth into account, so prefixes are weighted according to how much Tor capacity they contain.
As of August 2018 the Tor network consists of over 3100 unique IPv4 (and more than 200 IPv6) BGP prefixes located in over 1100 unique Autonomous Systems (ASes). The resiliency of the Tor related BGP prefixes has an influence on how hard it would be to actually hijack Tor traffic using BGP based attacks.
It is important to note that we only look at the prefixes that contain Tor relays but that is only half of the relevant part since an attacker could also hijack the BGP prefixes containing Tor clients or destination servers. Hijacking prefixes that contain a big fraction of the Tor network would probably be more efficient for an attacker because this would give them traffic of more Tor clients at once. The main disadvantage for the attacker: BGP hijacking is usually visible and not stealthy.
How resilient are the BGP prefixes containing Tor relays and what properties do we consider?
We categorize prefixes by analyzing the following properties
BGP Prefix Length
The prefix length is relevant in the context of hijacks because the longest prefix wins. This has an impact on whether a more-specific prefix hijacking attack is feasible. If you/your ISP is announcing /22 prefixes but the attacker announces the same space using multiple more-specific /24 prefixes his routes will generally and globally win.
IPv4 prefixes longer than /24 (/48 for IPv6) usually are not propagated on the internet. Even if you announce /24 the attacker can still hijack parts of the traffic depending on who is closer to the victim.
We measure the amount of Tor capacity located in /24 (IPv4) and /48 (IPv6) prefixes.
CAIDA AS Rank
We further sub-categorize each /24 (/48) prefix by looking at the CAIDA AS rank of the AS that announces the prefix. This should give us a rough estimate of whether their connectivity is better or worse than the median AS. We assume a threat model with an attacker in a median ranked AS. The median value for CAIDA’s AS rank is around 12500, we consider all ASes with an AS rank <= 10000 to be “better connected than the attacking AS” and have a shorter AS PATH than the attacking AS (so they are expected to “win the race” against the attacker).
RPKI Validity State (ROAs)
The resource public key infrastructure (RPKI) provides signed data about which AS is authorized to announce what prefixes (and optional max prefix length). Organizations holding IP space can authorize ASes to announce their prefixes. In RPKI the regional internet registries (RIRs) are the root CAs.
The available options to IP address holders (hosted vs. delegated PKI) depend on the RIR. Creating a ROA can be as easy as clicking a few buttons in a LIR interface (BGP router configuration changes are not required).
The RPKI adoption rate on the general internet is considered low, NIST’s RPKI monitor (figure 1) shows that RPKI adoption is around 10% when measured by address space,
but the RPKI adoption is steadily growing (especially in the RIPE region) as can be seen on RIPE’s RPKI statistics:
We aim to answer the question: How big is the RPKI ROA adoption for Tor relay prefixes?
Important note: The existence of a ROA alone does not make the covered prefixes magically more secure on its own. In addition to create ROAs, others also need to validate routes against ROAs by performing route origin validation (ROV). The ROV adoption rate is lower than the adoption rate of ROAs but some big internet exchanges (IXes) started to perform ROV on their route servers and so all customers using them benefit from created ROAs even if they do not perform ROV themselves.
It is hard to gather metrics on how much of the internet is actually performing ROV. The website rov.rpki.net shows a list of ASes performing ROV (with a certain likelihood) but that is only a partial view. In the context of Tor ROAs can be useful even if only a small fraction of ISPs are validating routes, more on that later.
ROA maxlength attribute
When creating a ROA, the IP holder can specify an optional max length field. This allows an AS to announce prefixes that are longer (more-specifics) than the prefix in the ROA, but it also introduces a potential attack opportunity when not all prefixes that are allowed by the ROA are actually announced. If the actual AS announces a /22 only but specifies a max length field of /24 in the ROA, then attackers can announce more-specific /24 prefixes (using the victim’s AS number) without causing RPKI INVALID validation results.
Due to the fact that the maxlength attribute is so often used insecurely, a current Internet-Draft in the IETF SIDROPS working group covers this topic and discourages the use of the maxlength attribute in ROAs.
We will look into what portion of the Tor capacity is covered by a ROA with a proper (minimal) vs. weak maxlength attribute.
RIPE IRR coverage (in-region only)
Internet routing registries (IRRs) also provide data to verify BGP routes (prefix to AS mapping) but their data quality varies. We consider only the IRR which has somewhat reasonable data quality and authorization checks in place: RIPE
In addition to limiting ourselves to RIPE’s IRR we further filter RIPE IRR entries to RIPE managed IP space to mimic what will be the case starting with 2018–09–04 anyway, because out of region entries are less trustworthy.
Results
51% of the Tor consensus weight fraction is not located in a /24 prefix and does not have a valid ROA. This portion of the Tor network is expected to be more vulnerable to hijacking attacks than the rest.
32% of the Tor capacity has a valid ROA (that is a much higher rate when compared with the overall internet). Unfortunately 12% of the Tor capacity is located in prefixes that have a weak maxlength in their ROA (maxlength is longer than the actual announced prefix). If wonder about a per-RIR breakdown of that 32% RPKI ROA portion: It is basically all RIPE (0.4% is from non-RIPE regions).
19% of the Tor capacity is located in /24 prefixes and more-specific hijacking attacks are considered impossible against that portion, 10% of which is in ASes with an CAIDA AS rank ≤ 10000. 9% have a CAIDA rank bigger than 10000 (weaker).
2% of the Tor capacity has a valid ROA and is located in /24 prefixes.
We looked into the /24 prefix capacity to see if there is a significant difference between exit and guard capacity:
The Tor exit fraction located in /24 prefixes is much higher than the guard capacity. So hijacks against exit capacity might be harder than attacks against guard capacity. This confirms observations that exit operators are more likely to have their own prefixes.
The following graph shows what fraction of the Tor capacity has which RPKI validity state:
Only ten IPv4 prefixes were found to be RPKI ‘Invalid’ (50% due to ASN mismatch, 50% due to prefix length). Only 4 out of the 10 prefixes (containing one relay each) would become unreachable (no route) due to missing alternative (RPKI ‘valid’) routes in an environment that performs ROV. This does also mean that Tor clients and relays that are located in an AS that performs ROV (or connected to a route server performing ROV) can not directly reach these 4 relays, but only one of the affected relays has the guard flag.
RIPE IRR Coverage (in-region only)
Most of the Tor capacity has a proper route object in the RIPE database. After looking at above graph you might wonder: How much of the entire Tor capacity is located in the RIPE region regardless of their RIPE IRR state? What would be the theoretical maximum for the blue portion of the graph? The answer is: 84.7%, that is good and bad at the same time. Bad for Tor RIR diversity and maybe good for routing security since the RIPE region has the best IRR quality and the highest RPKI adoption rates.
IPv6 Prefixes
We also looked at the over 200 IPv6 prefixes used for the Tor network, but it is important to note that only about 25% of the Tor network capacity has an IPv6 address and IPv6 is only used for entering and exiting the Tor network at this point. To be able to compare figure 7 directly with figure 3 we extrapolated IPv6 CW fraction values to 100%.
The RPKI adoption rate on IPv6 is significantly higher than on IPv4 (at least for the Tor prefixes).
Unlike for IPv4 prefixes there have been no RPKI ‘Invalid’ IPv6 prefixes at all but there is also a significant portion with weak maxlength value.
Who are the biggest RPKI ROA adopters on the Tor network?
ROAs are not created by the AS but by the IP address space holder, but for most cases this is the same entity so we simplify above question by grouping by the Autonomous System. This list can be of use to relay operators (and others) as a reference list when talking to their ISPs about enabling ROAs.
+------------------------------+------+--------+
| as_name | CWfr | relays |
+------------------------------+------+--------+
| Hetzner Online GmbH | 7.02 | 284 |
| Online S.a.s. | 5.16 | 113 |
| myLoc managed IT AG | 2.02 | 41 |
| netcup GmbH | 1.73 | 50 |
| NForce Entertainment B.V. | 1.50 | 25 |
| Voxility S.R.L. | 1.06 | 14 |
| SOFTplus Entwicklungen GmbH | 0.81 | 15 |
| ISPpro Internet KG | 0.62 | 21 |
| I.C.S. Trabia-Network S.R.L. | 0.61 | 45 |
| SWITCH | 0.48 | 9 |
| Telenor Norge AS | 0.39 | 28 |
| Joshua Peter McQuistan | 0.37 | 5 |
| 1&1 Internet SE | 0.34 | 8 |
| Brass Horn Communications | 0.33 | 6 |
| True B.V. | 0.30 | 1 |
| Deutsche Telekom AG | 0.29 | 152 |
+------------------------------+------+--------+This list is not implying that these ASes created ROAs for all their prefixes (see next section).
Who are the biggest Tor related network operators not adopting ROAs (completely)?
+------------------------+-------+--------+
| as_name | CWfr | relays |
+------------------------+-------+--------+
| OVH SAS | 13.04 | 530 |
| Online S.a.s. | 8.42 | 231 |
| Joshua Peter McQuistan | 2.83 | 27 |
| Hetzner Online GmbH | 2.67 | 72 |
| DigitalOcean, LLC | 1.80 | 274 |
| FranTech Solutions | 1.41 | 35 |
+------------------------+-------+--------+OVH SAS does apparently not use RPKI at all yet but more than 500 Tor relays run on their network, so the RPKI coverage could be increased a lot by convincing a single ISP to create ROAs for their prefixes.
Since Online S.a.s. and Hetzner were also among the biggest operators actually creating ROAs we took a closer look at the prefixes that do not have a ROA to get an idea why that might be the case. The short answer is: Not all IP space is equal.
We found that all the Online S.a.s. and Hetzner prefixes that lack a ROA have something in common, namely they are in “status: LEGACY” according to RIPE DB. Creating ROAs for such IP blocks was not possible from the very beginning, but there are options to convert legacy blocks to ALLOCATED PA blocks and according to RIPE’s documentation you can also create ROAs for legacy IP blocks (but probably less conveniently).
We also found a single IP address holder (IP space spread across 3 ASNs) that contains 14% of the Tor exit capacity that does not use RPKI ROAs yet.
Recommendations for Tor Relay Operators
1. Monitor your relay’s BGP prefix for suspicious BGP activity and share alerts with tor-relays@lists.torproject.org
The easiest way to do so is to subscribe to your prefixes using BGPmon. You should practically get zero alerts.
2. Check the following properties of the prefixes you use(ideally even before ordering servers):
- prefix length and IRR state
- RPKI state
3. Ask your ISP/IP holder to create ROAs for the prefixes you use, if the ROA is currently missing.
4. Ensure the ROA creator is aware of the risks of the maxlength attribute and uses it accordingly (in the best case not at all)
5. Monitor the RPKI validity state of your prefixes (can also be done with bgpmon)
6. Ask your ISP to announce the IP space of your relays in /24 prefixes (/48 for IPv6) to avoid more-specific prefix hijacks (this makes sense even if you have ROAs in place due to the low ROV coverage)
7. If your relay uses IP addresses from the RIPE region: ask your provider to create route(6) objects matching the announcements if they are not present yet. You can use RIPEstat’s prefix routing consistency widget to check the current state (the “In RIS” and “RIPE IRR” columns should both say “yes”).
8. Be aware that “LEGACY” or “ERX” IP space might be less likely to get ROAs by your ISP
9. Enable IPv6 on your relays
“Virtual” Route Origin Validation in the Tor Context
The are two good reasons why Tor should care of relays located in RPKI ‘Invalid’ prefixes:
- It will eventually break the “the Tor network is a full mesh” assumption. Relays in such RPKI ‘invalid’ prefixes with no alternative valid route will not be reachable from ASes performing ROV, but the Tor network assumes that every relay can reach every other relay. When ROV breaks that assumption it is better to exclude these relays than to keep only partially reachable relays.
- An RPKI ‘Invalid’ route might as well be an actual BGP hijacking attempt and why not stop that?
The obvious place to enforce ROV for the Tor network would be the Tor directory authorities that would run RPKI validators and vote for relays accordingly. At this point this is no more than an idea.
The unsolved problem: AS Path Verification
We only looked at route origin (which AS announces what prefix?) but even which a hypothetical ROA and ROV coverage of 100% BGP routing attacks that spoof the AS origin itself via AS PATH are still possible. At the IETF 102 SIDROPS meeting Alexander Azimov presented (15min) (slides) a new Internet-Draft that aims totackling AS PATH verification using ASPA which as an attendee and Geoff Huston pointed out is similar to soBGP. ASPA builds on top of RPKI as well (like ROAs).
Key Take Aways
- ROA adoption in the context of Tor is higher than on the general internet due to some big hosters (used by many relays) having adopted RPKI
- ROA adoption could further be significantly increase even if a very limited number of entities enable them (due to of the centralization around big ISPs like OVH)
- legacy IP blocks are apparently the reason for a significant portion (>10%) of the Tor network not having ROAs yet
- RIPE’s IRR (limited to RIPE managed space) covers most of the Tor network’s capacity (77%)
- More than 84% of the Tor capacity uses RIPE managed IP space
- Hijacking exit capacity is probably harder than hijacking guard capacity (due to the increased use of /24 prefixes for exits)
- ROA adoption is higher for IPv6 than for IPv4 while IPv6 has less (none) RPKI ‘Invalid’ routes
- a significant portion of ROAs has a weak maxlength attribute (this confirms what others have reported as well)
- ROV could be performed by Tor directory authorities to cover all relays (by rejecting/flagging them) even if global ROV deployment rate is very low
- most of the Tor network capacity is not covered by RPKI ROAs and is not located in /24 prefixes
Future Work: BGP Monitoring for Tor Prefixes
Since using commercially available services like BGPmon are too expensive to monitor the entire Tor network with over 3100 prefixes (>40k USD / month) there is a need for other solutions.
The Counter-RAPTOR paper describes a BGP monitoring system to detect routing attacks on the Tor network in near real-time (hourly) and some information surfaced on the Tor Project bug tracker but it does not appear to be deployed in practice and does not use RIPE’s IRR and RPKI ROAs as additional inputs. ARTEMIS is not a publicly available tool either. Available tools like tabi appear to have lots of false positives.
Acknowledgements
We’d like to explicitly mention the used data sources:
- Tor Project’s onionoo from 2018–08–09 21:00 UTC for relay IP addresses, cw fraction, guard probability and exit probability data
- RIPEstat for BGP prefix, ASN and IRR data. (We stumbled on a bug in RIPEstat’s IRR related part that might has a minor impact on the graph in figure 6 but we believe it to affect only a non-significant portion <1% cw fraction)
- RPKI Validator v2.24 (local instance with ARIN’s TAL enabled)
- RIPE IP Space to filter IRR entries to RIPE in-region items only
- CAIDA AS Rank API (2018–07–01)
Appendix
Mutually Agreed Norms for Routing Security (MANRS) by the Internet Society
