Are you an ISP in texas? Are you looking for better connectivity? Come visit midwest-ix for our new partnership in texas-ix at the US mum in Austin this coming Thursday and Friday (April 4th and 5th 2019).
If you haven’t registered today go over to the MidWest-IX facebook page and send us a message for your ticket, which includes free meals. You don’t get the free lunches with the standard ticket.
#usmum2019 #peering #keeptrafficlocal
Justin to be at WISPAMERICA 2019
Visit FD-IX at WISPAmerica 2019.
The folks over at PeeringDB have a presentation on setting up an account and adding your network.
NEW Service: IP Space Compliance
MTIN is announcing a new service today for those of you who have Registry assigned IP space. For the low price of $80 a year, MTIN will provide the following services in regards to your IP allocations.
- Make sure your whois information is correct in the proper registry (ARIN, APNIC, and others) each year.
- Make sure your peering db entries are correct and up-to-date
- Monitor your IP blocks for hijacking and other activity.
- Recommend any security changes needed.
- Setup peering sessions to Qrator for auditing purposes
- Monitor reverse DNS on your blocks for proper functionality
Unlimited of adding new IP blocks. The first registry or peeringdb change free each year.
If you need help setting up peering DB we can add a small setup to assist with this. contact firstname.lastname@example.org for details on this new service today.
Many ISPs run into this problem as part of their growing pains. This scenario usually starts happening with their third or 4th peer.
Scenario. ISP grows beyond the single connection they have. This can be 10 meg, 100 meg, gig or whatever. They start out looking for redundancy. The ISP brings in a second provider, usually at around the same bandwidth level. This way the network has two pretty equal paths to go out.
A unique problem usually develops as the network grows to the point of peaking the capacity of both of these connections. The ISP has to make a decision. Do they increase the capacity to just one provider? Most don’t have the budget to increase capacities to both providers. Now, if you increase one you are favouring one provider over another until the budget allows you to increase capacity on both. You are essentially in a state where you have to favor one provider in order to keep up capacity. If you fail over to the smaller pipe things could be just as bad as being down.
This is where many ISPs learn the hard way that BGP is not load balancing. But what about padding, communities, local-pref, and all that jazz? We will get to that. In the meantime, our ISP may have the opportunity to get to an Internet Exchange (IX) and offload things like streaming traffic. Traffic returns to a little more balance because you essentially have a 3rd provider with the IX connection. But, they growing pains don’t stop there.
As ISP’s, especially WISPs, have more and more resources to deal with cutting down latency they start seeking out better-peered networks. The next growing pain that becomes apparent is the networks with lots of high-end peers tend to charge more money. In order for the ISP to buy bandwidth they usually have to do it in smaller quantities from these types of providers. This introduces the probably of a mismatched pipe size again with a twist. The twist is the more, and better peers a network has the more traffic is going to want to travel to that peer. So, the more expensive peer, which you are probably buying less of, now wants to handle more of your traffic.
So, the network geeks will bring up things like padding, communities, local-pref, and all the tricks BGP has. But, at the end of the day, BGP is not load balancing. You can *influence* traffic, but BGP does not allow you to say “I want 100 megs of traffic here, and 500 megs here.” Keep in mind BGP deals with traffic to and from IP blocks, not the traffic itself.
So, how does the ISP solve this? Knowing about your upstream peers is the first thing. BGP looking glasses, peer reports such as those from Hurricane Electric, and general news help keep you on top of things. Things such as new peering points, acquisitions, and new data centers can influence an ISPs traffic. If your equipment supports things such as netflow, sflow, and other tools you can begin to build a picture of your traffic and what ASNs it is going to. This is your first major step. Get tools to know what ASNs the traffic is going to You can then take this data, and look at how your own peers are connected with these ASNs. You will start to see things like provider A is poorly peered with ASN 2906.
Once you know who your peers are and have a good feel on their peering then you can influence your traffic. If you know you don’t want to send traffic destined for ASN 2906 in or out provider A you can then start to implement AS padding and all the tricks we mentioned before. But, you need the greater picture before you can do that.
One last note. Peering is dynamic. You have to keep on top of the ecosystem as a whole.
As a service provider you have a mountain of terms to deal with. As you dive into the realm of BGP, you will hear many terms in regards to peers. Knowing their names AND your definition of them will serve you well. I emphasized the and in the last sentence because many people have different definitions of what these terms means. This can be due to how long they have been dealing with networks, what they do with them, and other such things. For example, many content providers use the term transit differently than an ISP. So, let’s get on to it.
Transit or upstream
This is what you will hear most often. A transit peer is someone who you go “through” in order to reach the internet. You transit their network to reach other networks. Many folks use the term “upstream provider” when talking about someone they buy their internet from.
Someone who is “downstream” is someone you are providing Internet to. They are “transiting” your network to reach the Internet. This is typically someone you are selling Internet to.
This is the term which probably needs the most clarification when communicating with others about how your BGP is setup. A peer is most often used as a generic term, much like Soda (or pop depending on where you are from). For example someone could say:
“I have a peer setup with my upstream provider who is Cogent.” This is perfectly acceptable when used with the addition of “my upstream provider”. Peers are often referred to as “neighbors” or “BGP neighbors”.
Local or Private Peer
So what is a local peer? A local peer is a network you are “peering” with and you are only exchanging routes which are their own or their downstream networks. A local peer usually happens most often at an Internet Exchange (IX) but can happen in common points where networks meet. The most important thing that defines a local peer is you are not using them to reach IP space which is not being advertised form their ASN. Your peering relationship is just between the two of you. This gets a little muddy when you are peering on an IX, but thats being picky.
I have trained myself to qualify what I mean by a peer when talking about them. I will often say a “transit peer” or a “local peer”. This helps to add a little bit of clarity to what you mean.
Why is this all important? For one, it helps with keeping everyone on the same page when talking about peering. I had a case a few weeks ago where a Content provider and I wasted configuration time because our definition of transit was different. Secondly, you want to be able to classify your peers so you can apply different filter rules to them. For example, with a downstream peer you only want to accept the IP space they have shown you which is their own. That way you are not sending your own transit traffic over their network. This would be bad. However, if you are accepting full routes from your transit provider, you want your filters to accept much more IP than a downstream provider. So if you have a team being able to be on the same page about peers will help when it comes to writing filters, and how your routers “treat” the peer in terms of access lists, route filters, etc.
Internet Routing Registries
Routing Registries are a mysterious underpinning of the peering and BGP world. To many they are arcane and complicated. If you have found this article you are at least investigating the use of a registry. Either that or you have ran out of fluffy kittens to watch on YouTube. Either way one of the first questions is “Why use a routing registry”.
As many of us know BGP is a very fragile ecosystem. Many providers edit access lists in order to only announce prefixes they have manually verified someone has the authority to advertise. This is a manual process for many opportunities for error. Any time a config file is edited errors can occur. Either typos, misconfiguration, or software bugs.
Routing registries attempt to solve two major issues. The first is automating the process of knowing who has authority to advertise what. The second is allowing a central repository of this data.
So what is a routing Registry?
From Wikipedia: An Internet Routing Registry (IRR) is a database of Internet route objects for determining, and sharing route and related information used for configuring routers, with a view to avoiding problematic issues between Internet service providers.
The Internet routing registry works by providing an interlinked hierarchy of objects designed to facilitate the organization of IP routing between organizations, and also to provide data in an appropriate format for automatic programming of routers. Network engineers from participating organizations are authorized to modify the Routing Policy Specification Language (RPSL) objects, in the registry, for their own networks. Then, any network engineer, or member of the public, is able to query the route registry for particular information of interest.
What are the downsides of a RR?
Not everyone uses routing registries. So if you only allowed routes from RR’s you would get a very incomplete view of the Internet and not be able to reach a good amount of it.
Okay, so if everyone doesn’t use it why should i go to the trouble?
If you are at a formal Internet Exchange (IX) you are most likely required to use one. Some large upstream providers highly encourage you to use one to automate their process.
What are these objects and attributes?
In order to partipate you have to define objects. The first one you create is the maintainer object. This is what the rest of the objects are referenced to and based from. Think of this as setting up your details in the registry.
From this point you setup “object types”. Object types include:
If you want to learn more about each of these as well as templates visit this ARIN site.
So what do I need to do to get started?
The first thing you need to do is setup your mntner object in the registry. I will use ARIN as our example. You can read all about it here:https://www.arin.net/resources/routing/.
You will need a couple of things before setting this up
1.Your ARIN ORGID
2.Your ADMIN POC for that ORGID
3.Your TECH POC for that ORGID
Once you have these you can fill out a basic template and submit to ARIN.
descr: Example, Inc.
auth: MD5-PW $1$ucVwrzQH$zyamFnmJ3XsWEnrKn2eQS/
changed: email@example.com 20150202
The templates is very specific on what to fill out. The mnt-by and referral-by are key to following instructions. MD5 is another sticking point. The process is documented just in a couple of places. In order to generate your MD5-PW follow these instructions.
1.Go to https://apps.db.ripe.net/crypt/ Enter in a password. Make sure you keep this cleartext password as you will need it when sending future requests to ARIN’s Routing Registry.
2.Submit the password to get the md5 crypt password. Keep this password for your records, as you may need it when interacting with ARIN’s IRR in the future.
3.Add the following line to your mntner object template in the text editor.
Our example above has a MD5 password already generated.
Once this is done and created you can add objects. The most commonly added objects are your ASN and IP space.
Create your ASN object using the as-num template
descr: Example, Inc.
descr: 114 Pine Circle
descr: ANYWHERE, IN 12345
import: from AS65535 accept ANY
import: from AS65533 accept AS65534
export: to AS65533 announce ANY
export: to AS65535 announce AS2 AS65533
changed: firstname.lastname@example.org 20150202
The things to know about the above template are the import and export attributes.
Now on to adding IP space
Suppose you have IP space of 192.0.2.0/24 Your template would look like:
inetnum: 192.0.2.0 – 192.0.2.255
descr: Example, Inc.
descr: 115 Oak Circle
descr: ANYWHERE, IN 12345
changed: email@example.com 20150202
The password attribute is the cleartext password for your MD5 key.
Using RPSL in practice
Case Study from MidWest-IX
Outlines a real world example of how an exchange benefits a network operator