ethernet MTU and overhead

One of the most common questions is how much overhead do I need to account for on my transport network? I have put together a quick list to help when you are calculating your overhead.

-GRE (IP Protocol 47) (RFC 2784): 24 bytes (20 byte IPv4 header, 4 byte GRE header)
-6in4 encapsulation (IP Protocol 41, RFC 4213): 20 bytes
-4in6 encapsulation (e.g. DS-Lite RFC 6333): 40 bytes
Addition IPv4 header:20 bytes
-IPsec encryption:
73 bytes for ESP-AES-256 and ESP-SHA-HMAC overhead (overhead depends on transport or tunnel mode and the encryption/authentication algorithm and HMAC)
-MPLS: 4 bytes for each label in the stack
-802.1Q tag: 4 bytes
Q-in-Q: 8 bytes
-VXLAN: 50 bytes
-OTV: 42 bytes

Some rules of thumb when setting MTUs. You won’t get fragmentation if your L2 MTU is higher than your L3 MTU. This is just not the setting, but the actual overhead in use. Just setting it to a number doesn’t necessarily make it right. The above list will help you calculate the minimum MTU you may need. I try to get gear that supports a 1548 MTU and set everything to that. Makes it simple. I still want to know how much MTU I am utilizing because it helps me validate my designs.
The most important rule of thumb is you won’t get fragmentation if your l3 MTU is less than your L2 MTU.

BGP local Pref and you

One of the bgp topics that comes up from time to time is what does “bgp local-pref” do for me? The short answer is it allows you to prefer which direction a traffic will flow to a given destination. How can this help you? Well before we start, remember the high number wins in local-pref.
Let’s assume you are an ISP. You have the following connections:
-You supply a BGP connection to a downstream client.
-You have a private peer setup with the local college
-You are hooked into a local internet exchange
-You have transport to another internet exchange in the next state over
-and you have some transit connections where you buy internet.

So how do we use BGP preference to help us out? We might apply the following rules to routes received from our various peers
Our downstream client we might set their local pref to 150
The college we may set them to 140
Preferred internet exchange peering: 130
Next state IX: 120
Transit ISPs: 100

Now these don’t make much sense by themselves, but they do when you take into account how BGP would make a decision if it has to choose between multiple paths. If it only has one path to a certain route then local-pref is not relevant.

Let’s say you have a customer on your ISP that is sending traffic to a server at a local college. Maybe they are a professor who is remoting into a server at the college to run experiments. There are probably multiple ways for that traffic to go. If the college is on the local Internet exchange you are a member of, that is one route, the next route would be your transit ISPs, and obviously your private peer with the college. So, in our example above the college, with a local pref of 140 wins out over the local exchange, wins out over the next state IX, and wins out over the Transit ISPs. We want it to go direct over the direct peer with the college. Mission accomplished.

local-pref is just one way to engineer your traffic to go out certain links. Keep in mind two things:
1.Higher number wins
2.local-pref only matters if there are multiple paths to the same destination.
3.Local-pref has to do with outbound path selection