Quick Reference: OSPF Network Types

Point-to-multipoint is treated as a collection of point-to-point links and thus no DR/BDR is required.

Point-to-Point is a single link and no election is needed.

Broadcast: OSPF routers on broadcast networks will elect a DR and a BDR (since it is multiaccess) – OSPF packets are multicast.

NBMA: Routers will elect DR and BDR (since it is multiaccess), but since it is a non-broadcast, routers will have to communicate via unicast rather than multicast.

12 days of netmas

On the first day of netmas
my true love sent to me:
A spanning tree instance

On the second day of netmas
my true love sent to me:
2 ethernet ends
and a spanning tree instance

On the third day of netmas
my true love sent to me:
3 sfps
2 ethernet ends
and a spanning tree instance

On the fourth day of netmas
my true love sent to me:
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the fifth day of netmas
my true love sent to me:
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the sixth day of netmas
my true love sent to me:
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the seventh day of netmas
my true love sent to me:
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the eighth day of netmas
my true love sent to me:
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the ninth day of netmas
my true love sent to me:
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the tenth day of netmas
my true love sent to me:
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the eleventh day of netmas
my true love sent to me:
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the twelveth day of netmas
my true love sent to me:
12 routers on a stick
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

Learning, certifications and the xISP

One of the most asked questions which comes up in the xISP world is “How do I learn this stuff?”.   Depending on who you ask this could be a lengthy answer or a simple one sentence answer.  Before we answer the question, let’s dive into why the answer is complicated.

In many enterprise environments, there is usually pretty standard deployment of networking hardware.  Typically this is from a certain vendor.  There are many factors involved. in why this is.  The first is total Cost of Ownership (TCO).  It almost always costs less to support one product than to support multiples.  Things like staff training are usually a big factor.  If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.

Another factor involved is economies of scale.  Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words.  You can commit to buying product over time or all at once.

So, to answer this question in simple terms.  If your network runs Mikrotik, go to a Mikrotik training course.  If you run Ubiquiti go to a Ubiquiti training class.

Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario.  Many of our xISP clients have gear from several vendors deployed.  They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between.  So where does a person start?

We recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.

1.Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S).  There are a ton of ways to study for this certification.   There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of studying for this certification, you will learn many things which will carry over to any vendor.  Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks.  During the course of studying you will learn, and then reinforce that through practice tests and such.  Don’t necessarily focus on the goal of passing the test, focus on the content of the material.  I used to work with a guy who went into every test with the goal of passing at 100%.  This meant he had to know the material. CompTIA is a side path to the Cisco CCNA.  For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.

2.Once you have the CCNA under your belt, take a course in a vendor you will be working the most with.  At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.

3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook.  Think of your networking knowledge as your end goal is to be able to build a house.  Steps one and two have given you general knowledge, you can now use tools, do some basic configuration.  But you can’t build a house without knowing what is involved in designing foundations,  what materials you need to use, how to compact the soil, etc.  Network design is no different. These are not things you can read in a manual on how to use the tool.  They also are not tool specific.   Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track.  Just follow my philosophy in relationship to #1. Focus on the material.

Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is a key to your success.  If you understand what goes on with an IP packet, learning tools like Wireshark will be easier.  As you progress let things grow organically from this point.  Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options.  Branch out into some other areas ,such as security, to add to your overall understanding.

Never stop learning! Visit our online store for links to recommend books and products.

WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not.  My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
Butch Evans
Dennis Burgess
Rickey Frey
Steve Discher
Baltic Networks

Vendor Certification Pages
Ubiquiti
Mikrotik
Cisco
Juniper
CWNA
CompTIA

If you provide training let me know and I will add you to this list.

pfSense 2.3 released

From the folks over at pfSense

We are happy to announce the release of pfSense® software version 2.3!

The most significant changes in this release are a rewrite of the
webGUI utilizing Bootstrap, and the underlying system, including the
base system and kernel, being converted entirely to FreeBSD pkg. The
pkg conversion enables us to update pieces of the system individually
going forward, rather than the monolithic updates of the past. The
webGUI rewrite brings a new responsive look and feel to pfSense
requiring a minimum of resizing or scrolling on a wide range of
devices from desktop to mobile phones.

You can find all the details in the release announcement here:

pfSense 2.3-RELEASE Now Available!

Thanks for your support

How does BGP select which route?

BGP can be a complex and almost mystical protocol. For those of you who are trying to determine how BGP selects which route here is your guide. Before we get into it a couple of things to keep in mind. First, BGP is not a multipath routing protocol. This is different than what you may be used to with OSPF. BGP goes to great lengths to encure only one route is used. Secondly, there are some vendor specific rules which are applied. I will try to point those out as we go along.

1.The first test is if the next hop router is accessible.

2.If Synchronization is enabled, the router will ignore any iBGP routes which are not synced.

3.The third is Cisco specific. Cisco uses a weight attribute. The largest weight wins. Default weight is zero. Maximum weight is 65,535.

4.If the weights are the same, the highest local preference is chosen from LOCAL_PREF. It’s important to note that routers only receive this from iBGP.

5.Net up, the router checks to see if any of the possible routes were originated locally. The two main checks are either the network or aggregate commands. The network command wins if it is originated locally.

6.If two or more routes are still equal the router looks as AS_PATH. The router will prefer any iBGP routes. Outside of the AS BGP will prefer the shortest path.

7.BGP then moves on to the ORIGIN attribute. If the path lengths are the same, BGP selects IGP over EGP and EGP over INCOMPLETE routes.

8.BGP now looks at MED values. The lowest value is selected. Note, MED is only used if both routes are received from the same AS, or if always-compare-med has been enabled. Be careful with always-compare-meds as this can cause routing loops.

9.BGP will then prefer eBGP to iBGP routes. This is not the same as #5 above. Only external routes are looked at here.

10.Next IBGP costs are compared to the next hop routers. The closest one is selected.

11.Ages of routes are finally connected. This is kind of like choosing teams for dodgeball. The oldest route wins. The reason being is oldest routes are thought to be more stable.

12.And finally, if all else fails the router with the lowest router ID wins.

This is a quick low-down on how BGP “thinks” in order to determine routes. If anyone has some Cisco, Mikrotik, quagga, or other specific attributes please comment. I have reached out to Mikrotik and Ubiquiti specifically to see if this is in-line with their implementation of BGP.

Helpful OSPF times

OSPF can be a mystery to some.  Understanding the default timeouts can be helpful in troubleshooting.  Some vendors change these times so it is very important to realize this stuff if you start mixing vendors in your OSPF domains.

10 Seconds
Default OSPF hello timer on broadcast and point-to-point links

30 Seconds
Default OSPF hello timer on nonbroadcast links

40 Seconds
Default OSPF hold timer on broadcast and point-to-point links

120 Seconds
Default OSPF hold timer on nonbroadcast links

30 Minutes
OSPF LSA refresh timer

60 Minutes
OSPF LSA expiration timer

 

 

OSFP and areas

OSPF areas are one of the more common topics I am asked about as networks grow.   Before we dig into this, we need to understand the reasons why OSPF areas were created in the first place.  Next, we will go into how to apply areas to modern network designs.

Why did areas come into being?
Let’s rewind to a time where RAM in routers was very expensive.  Processors were expensive.  One of the biggest reasons OSPF areas came into play was ram limitations. Every route in your routing table takes up ram.  The more routes you have the more ram that is taken up in each router to hold that table.  Also, in order to calculate these routes processor power is used up.

So what do areas do for me?
OSPF areas have many advantages.  However, in most WISP and ISP networks they serve two purposes.

The first purpose is they group similar devices into logical groups.  These groups can have filtering policies applied to them.

The second purpose is more important.  Implementing areas reduces the size of the routing table.  By doing this your routers spend less time calculating routes, and less time updating the database during a topology change. By reducing the routing table you also speed up what is called convergence time.  This is the time the entire network needs to agree on the current routing topology.   If a major backbone link is flapping your routers could be spending a fair amount of their resources calculating routing tables.

An important thing to note with implementing areas is you must have a good IP network design.   This means your sites/pops/towers should have a logical design which allows for easy route summarization and consolidation.  If you are looking into areas make sure you can summarize your routes in that area easily.  The following examples illustrate why this is important.

ospf1

 

In the above example we have created “Area1”.  Since we have a good IP network design we can summarize our routing table in and out of area1 into a few different ways.  If we want to reserve the whole 10.5.0.0/16 for future expansion then we can.  Or if we want to break this down into 10.5.0.0/20 or even smaller we can.  Part of this depends on growth plans.  With areas you have to keep in mind every area needs to touch the backbone (area 0.0.0.0) directly.  Now, you can use Virtual Links to have one non-backbone area traverse another non-backbone area.  However, even though is a standard, is a workaround at best.  There are many disadvantages to virtual links.

Now, back to our example.  If we create an area the 10.6.1.0/24 and 10.6.2.0/24 routers are the “in roads” to Area1.  These are known as area border routers (ABR).The main function of ABRs is to summarize sub networks found throughout the OSPF network. It stores many copies of its link-state database in memory when one of the stored copies shows an area where the actual router is connected. The ABR holds a minimum of two copies of the routing tables.  One from the backbone area, and one from each area it is connected to.

But, I thought areas were supposed to cut down on ram and CPU usage? Well, everything has a tradeoff.  This is where the philosophy side of things come into play, and probably the reason you have read this far.

When, how, and should I implement areas?
In today’s modern world with fast ram, fast links, and fast processors OSPF areas are needed less and less.  Routers today have more ram than even 5 years ago.  This means they can hold larger routing tables and do more calculations.

If you are thinking about implementing areas the first thing to look at is your IP design.  In order to take the best advantage of areas you should have a logical, and congruent design.  What I mean by this is your towers should be able to summarized as much as possible.  If you can fit 20 towers into a single route statement that is one good place an area would make sense.  If those 20 towers are not able to be summarized then adding an area is not going to be much of a benefit to you.

Network size does not necessarily dictate the need for OSPF areas.  If you have a neatly summarized IP network the need for areas is lessened.

What about if you are trying to join two different networks?
Say you purchased a neighboring ISP and want to join the new network with your own.   If you have overlapping IP space then things might not mesh together well, even with areas.   Most times you are better off running BGP with the two separate networks.  This allows each network to have it’s own space, own routing policies, but still be able to share bandwidth and other resources.  You simply don’t announce any overlapping space to each network until things are re-numbered.

One question I get in this scenario is my router can’t handle BGP.  BGP is a fairly lightweight protocol.  The issues arise when you start pulling in full or partial internet routing tables.  This is the same concept as mentioned above with the OSPF routes.

Internet Routing Registries

Routing Registries are a mysterious underpinning of the peering and BGP world. To many they are arcane and complicated. If you have found this article you are at least investigating the use of a registry. Either that or you have ran out of fluffy kittens to watch on YouTube. Either way one of the first questions is “Why use a routing registry”.

As many of us know BGP is a very fragile ecosystem. Many providers edit access lists in order to only announce prefixes they have manually verified someone has the authority to advertise. This is a manual process for many opportunities for error. Any time a config file is edited errors can occur. Either typos, misconfiguration, or software bugs.

Routing registries attempt to solve two major issues. The first is automating the process of knowing who has authority to advertise what. The second is allowing a central repository of this data.

So what is a routing Registry?
From Wikipedia: An Internet Routing Registry (IRR) is a database of Internet route objects for determining, and sharing route and related information used for configuring routers, with a view to avoiding problematic issues between Internet service providers.

The Internet routing registry works by providing an interlinked hierarchy of objects designed to facilitate the organization of IP routing between organizations, and also to provide data in an appropriate format for automatic programming of routers. Network engineers from participating organizations are authorized to modify the Routing Policy Specification Language (RPSL) objects, in the registry, for their own networks. Then, any network engineer, or member of the public, is able to query the route registry for particular information of interest.

What are the downsides of a RR?
Not everyone uses routing registries. So if you only allowed routes from RR’s you would get a very incomplete view of the Internet and not be able to reach a good amount of it.

Okay, so if everyone doesn’t use it why should i go to the trouble?
If you are at a formal Internet Exchange (IX) you are most likely required to use one. Some large upstream providers highly encourage you to use one to automate their process.

What are these objects and attributes?
In order to partipate you have to define objects. The first one you create is the maintainer object. This is what the rest of the objects are referenced to and based from. Think of this as setting up your details in the registry.

From this point you setup “object types”. Object types include:
as-set
aut-num
inet6num
inetnum
inet-rtr
key-cert
mntner
route
route6
route-set
If you want to learn more about each of these as well as templates visit this ARIN site.

So what do I need to do to get started?
The first thing you need to do is setup your mntner object in the registry. I will use ARIN as our example. You can read all about it here:https://www.arin.net/resources/routing/.

You will need a couple of things before setting this up
1.Your ARIN ORGID
2.Your ADMIN POC for that ORGID
3.Your TECH POC for that ORGID

Once you have these you can fill out a basic template and submit to ARIN.

mntner: MNT-YOURORGID
descr: Example, Inc.
admin-c: EXAMPLE123-ARIN
tech-c: EXAMPLE456-ARIN
upd-to: hostmaster@example.net
mnt-nfy: hostmaster@example.net
auth: MD5-PW $1$ucVwrzQH$zyamFnmJ3XsWEnrKn2eQS/
mnt-by: MNT-YOURORGID
referral-by: MNT-YOURORGID
changed: hostmaster@example.net 20150202
source: ARIN

The templates is very specific on what to fill out. The mnt-by and referral-by are key to following instructions. MD5 is another sticking point. The process is documented just in a couple of places. In order to generate your MD5-PW follow these instructions.

1.Go to https://apps.db.ripe.net/crypt/ Enter in a password. Make sure you keep this cleartext password as you will need it when sending future requests to ARIN’s Routing Registry.
2.Submit the password to get the md5 crypt password. Keep this password for your records, as you may need it when interacting with ARIN’s IRR in the future.
3.Add the following line to your mntner object template in the text editor.
auth: MD5-PW
Our example above has a MD5 password already generated.
Once this is done and created you can add objects. The most commonly added objects are your ASN and IP space.

Create your ASN object using the as-num template

aut-num: AS65534
as-name: EXAMPLE-AS
descr: Example, Inc.
descr: 114 Pine Circle
descr: ANYWHERE, IN 12345
descr: US
import: from AS65535 accept ANY
import: from AS65533 accept AS65534
export: to AS65533 announce ANY
export: to AS65535 announce AS2 AS65533
admin-c: EXAMPLE456-ARIN
tech-c: EXAMPLE123-ARIN
mnt-by: MNT-YOURORGID
changed: user@example.com 20150202
source: ARIN
password:

The things to know about the above template are the import and export attributes.

Now on to adding IP space
Suppose you have IP space of 192.0.2.0/24 Your template would look like:

inetnum: 192.0.2.0 – 192.0.2.255
netname: EXAMPLE-NET
descr: Example, Inc.
descr: 115 Oak Circle
descr: ANYWHERE, IN 12345
country: US
admin-c: EXAMPLE123-ARIN
tech-c: EXAMPLE456-ARIN
notify: user@example.com
mnt-by: MNT-YOURORGID
changed: user@example.com 20150202
source: ARIN
password:

The password attribute is the cleartext password for your MD5 key.

Further Reading:
Using RPSL in practice

NANOG IRR

BGP lockdown hints

As I am preparing talks for the upcoming WISPAPALOOZA 2014 in Las Vegas I am making some notes on advanced BGP.  If you are running BGP, and want to lock it down a little here are some general hints.  If you want more attend my session in Vegas or look here afterwords for the full rundown.

General Hints for BGP filter.

1.Filter all all the bogon addresses unless you have a specific need. If you have to ask you probably don’t have a need so filter it. Bogons are:
10.0.0.0/8,
172.16.0.0/12
192.168.0.0/16
169.254.0.0/16

2.Don’t accept your own IP space from upstreams.  There should be no reason someone is advertising your own IP space back to you that is not a downstream customer.  I mean dowstream as to someone you have assigned your own IP space to.

3.Limit the maximum number of prefixes your router will accept.

4.Most ISPs don’t announce anything less than a /24.  Configure your filters to not accept anything smaller than a /24 unless you have a specific need to do so.

5. Separate iBGP from eBGP.

6.Understand the defaults for the platform you are using.