For those of you not able to attend the US MUM presentation here is my presentation slides in PDF for my BGP session. 200 meg download.
Visit FD-IX at WISPAmerica 2019.
The folks over at PeeringDB have a presentation on setting up an account and adding your network.
MTIN is announcing a new service today for those of you who have Registry assigned IP space. For the low price of $80 a year, MTIN will provide the following services in regards to your IP allocations.
- Make sure your whois information is correct in the proper registry (ARIN, APNIC, and others) each year.
- Make sure your peering db entries are correct and up-to-date
- Monitor your IP blocks for hijacking and other activity.
- Recommend any security changes needed.
- Setup peering sessions to Qrator for auditing purposes
- Monitor reverse DNS on your blocks for proper functionality
Unlimited of adding new IP blocks. The first registry or peeringdb change free each year.
If you need help setting up peering DB we can add a small setup to assist with this. contact email@example.com for details on this new service today.
This document defines the BGP Monitoring Protocol (BMP), which can be used to monitor BGP sessions. BMP is intended to provide a convenient interface for obtaining route views. Prior to the introduction of BMP, screen scraping was the most commonly used approach to obtaining such views. The design goals are to keep BMP simple, useful, easily implemented, and minimally service affecting. BMP is not suitable for use as a routing protocol.
I am starting an ongoing series involving a semi-static set of devices. These will involve different tutorials on things such as OSPF, cambium configuration, vlans, and other topics. Below is the general topology I will use for this lab network. As things progress I will be able to swap different manufacturers and device models into this scenario without changing the overall topology. We may add a device or two here and there, but overall this basic setup will remain the same. This will allow you to see how different things are configured in the same environment without changing the overall scheme too much.
We will start with very basic steps. How to login to the router, how to set an IP address, then we will move to setting up a wireless bridge between the two routers. Once we have that done we will move onto setting up OSPF to enable dynamic routing. After that the topics are open. I have things like BGP planned, and some other things. If there is anything you would like to see please let me know.
Many ISPs run into this problem as part of their growing pains. This scenario usually starts happening with their third or 4th peer.
Scenario. ISP grows beyond the single connection they have. This can be 10 meg, 100 meg, gig or whatever. They start out looking for redundancy. The ISP brings in a second provider, usually at around the same bandwidth level. This way the network has two pretty equal paths to go out.
A unique problem usually develops as the network grows to the point of peaking the capacity of both of these connections. The ISP has to make a decision. Do they increase the capacity to just one provider? Most don’t have the budget to increase capacities to both providers. Now, if you increase one you are favouring one provider over another until the budget allows you to increase capacity on both. You are essentially in a state where you have to favor one provider in order to keep up capacity. If you fail over to the smaller pipe things could be just as bad as being down.
This is where many ISPs learn the hard way that BGP is not load balancing. But what about padding, communities, local-pref, and all that jazz? We will get to that. In the meantime, our ISP may have the opportunity to get to an Internet Exchange (IX) and offload things like streaming traffic. Traffic returns to a little more balance because you essentially have a 3rd provider with the IX connection. But, they growing pains don’t stop there.
As ISP’s, especially WISPs, have more and more resources to deal with cutting down latency they start seeking out better-peered networks. The next growing pain that becomes apparent is the networks with lots of high-end peers tend to charge more money. In order for the ISP to buy bandwidth they usually have to do it in smaller quantities from these types of providers. This introduces the probably of a mismatched pipe size again with a twist. The twist is the more, and better peers a network has the more traffic is going to want to travel to that peer. So, the more expensive peer, which you are probably buying less of, now wants to handle more of your traffic.
So, the network geeks will bring up things like padding, communities, local-pref, and all the tricks BGP has. But, at the end of the day, BGP is not load balancing. You can *influence* traffic, but BGP does not allow you to say “I want 100 megs of traffic here, and 500 megs here.” Keep in mind BGP deals with traffic to and from IP blocks, not the traffic itself.
So, how does the ISP solve this? Knowing about your upstream peers is the first thing. BGP looking glasses, peer reports such as those from Hurricane Electric, and general news help keep you on top of things. Things such as new peering points, acquisitions, and new data centers can influence an ISPs traffic. If your equipment supports things such as netflow, sflow, and other tools you can begin to build a picture of your traffic and what ASNs it is going to. This is your first major step. Get tools to know what ASNs the traffic is going to You can then take this data, and look at how your own peers are connected with these ASNs. You will start to see things like provider A is poorly peered with ASN 2906.
Once you know who your peers are and have a good feel on their peering then you can influence your traffic. If you know you don’t want to send traffic destined for ASN 2906 in or out provider A you can then start to implement AS padding and all the tricks we mentioned before. But, you need the greater picture before you can do that.
One last note. Peering is dynamic. You have to keep on top of the ecosystem as a whole.
Below, We have some visio diagrams we have done for customers.
This first design is a customer mesh into a couple of different data centers. We are referring to this as a switch-centric design. This has been talked about in the forums and switch-centric seems like as good as any.
This next design is a netonix switch and a Baicells deployment.
One of the bgp topics that comes up from time to time is what does “bgp local-pref” do for me? The short answer is it allows you to prefer which direction a traffic will flow to a given destination. How can this help you? Well before we start, remember the high number wins in local-pref.
Let’s assume you are an ISP. You have the following connections:
-You supply a BGP connection to a downstream client.
-You have a private peer setup with the local college
-You are hooked into a local internet exchange
-You have transport to another internet exchange in the next state over
-and you have some transit connections where you buy internet.
So how do we use BGP preference to help us out? We might apply the following rules to routes received from our various peers
Our downstream client we might set their local pref to 150
The college we may set them to 140
Preferred internet exchange peering: 130
Next state IX: 120
Transit ISPs: 100
Now these don’t make much sense by themselves, but they do when you take into account how BGP would make a decision if it has to choose between multiple paths. If it only has one path to a certain route then local-pref is not relevant.
Let’s say you have a customer on your ISP that is sending traffic to a server at a local college. Maybe they are a professor who is remoting into a server at the college to run experiments. There are probably multiple ways for that traffic to go. If the college is on the local Internet exchange you are a member of, that is one route, the next route would be your transit ISPs, and obviously your private peer with the college. So, in our example above the college, with a local pref of 140 wins out over the local exchange, wins out over the next state IX, and wins out over the Transit ISPs. We want it to go direct over the direct peer with the college. Mission accomplished.
local-pref is just one way to engineer your traffic to go out certain links. Keep in mind two things:
1.Higher number wins
2.local-pref only matters if there are multiple paths to the same destination.
3.Local-pref has to do with outbound path selection