Some Random Visio diagram

Below, We have some visio diagrams we have done for customers.

This first design is a customer mesh into a couple of different data centers. We are referring to this as a switch-centric design. This has been talked about in the forums and switch-centric seems like as good as any.

This next design is a netonix switch and a Baicells deployment.

Design for a customer

Where does Trill and VXLAN fit in your strategy?

As networking trends yo-yo between layer-3 and layer-2,  different protocols have emerged to address issues with large layer-2 networks. Protocols such as Transparent Interconnection of Lots of Links (TRILL), Shortest Path Bridging (SPB), and Virtual Extensible LAN (VXLAN) have emerged to address the need for scalability at Layer2.   Cloud scalability, spanning tree bridging issues, and big broadcast networks start to become a problem in a large data center or cloud environment.

To figure out if things like TRILL is a solution for you, you must understand the problem that is being addressed by TRILL. The same goes for the rest of the mentioned protocols. When it boils down to it the reason for looking at such protocols is you want high switching capacity, low latency, and redundancy.  The current de facto standard of Spanning Tree Protocol (STP) simply is unable to meet the needs of modern layer2 networks.  TRILL addresses the problem of STP’s ability to only allow one network path between switches or ports.  STP prevents loops by managing active layer -2 paths.   TRILL applies Intermediate System-to-Intermediate System protocol (IS-IS), which is a layer3 routing protocol translated to Layer 2 devices.

For those who say TRILL is not the answer things like SPB also known as 802.1aq, and VXLAN are the alternatives. A presentation at NANOG 50 in 2010 addressed some of the SPB vs TRILL debate. This presentation goes into great detail on the differences between the two.

The problem, which is one most folks overlook, is that you can only make a layer 2 network so flat.  The trend for a while, especially in data centers, is to flatten out the network. Is TRILL better? Is SPB better? The problem isn’t what is the better solution to use.  What needs to be addressed is the design philosophy behind why you need to use such things.   Having large Layer2 networks is generally a bad idea. Scaling issues can almost always be solved by Layer-3.

So, and this is where the philosophy starts, is TRILL, SPB, or even VXLAN for you? Yes, but with a very big asterisk. TRILL is one of those stop-gap measures or one of those targeted things to use in specific instances. TRILL reduces complexity and makes layer-2 more robust when compared to MLAG. Where would you use such things? One common decision of whether to use TRILL or not comes in a virtualized environment such as VSPHERE.

Many vendors such as Juniper, have developed their own solutions to such things.  Juniper and their Virtual Chassis solution do away with spanning tree issues, which is what TRILL addresses.   Cisco has FabricPath, which is Cisco’s proprietary TRILL-based solution. Keep in mind, this is still TRILL.   If you want to learn some more about Fabric Path this article by Joel Knight gets to the heart of Fabric path.

Many networks see VXLAN as their upgrade path.  VXLAN allows layer 2 to be stretched across layer 3 boundaries. If you are a “Microsoft person” you probably hear an awful lot about Network Virtualization using Generic Routing Encapsulation (NVGRE) which can encapsulate a layer two frame into IP.

The last thing to consider in this entire debate is how does Software Defined Networking (SDN) play into this. Many folks think controllers will make ECMP and MLAG easy to create and maintain. If centralized controllers have a complete view of the network there is no longer a need to run protocols such as TRILL.   The individual switch no longer makes the decision, the controller does.

Should you use Trill, VXLAN, or any of the others mentioned? If you have a large Layer-2 virtualized environment it might be something to consider.  Are you an ISP, there is a very small case for running TRILL in anything other than your data center. Things such as Carrier Ethernet and MPLS are the way to go.

12 days of netmas

On the first day of netmas
my true love sent to me:
A spanning tree instance

On the second day of netmas
my true love sent to me:
2 ethernet ends
and a spanning tree instance

On the third day of netmas
my true love sent to me:
3 sfps
2 ethernet ends
and a spanning tree instance

On the fourth day of netmas
my true love sent to me:
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the fifth day of netmas
my true love sent to me:
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the sixth day of netmas
my true love sent to me:
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the seventh day of netmas
my true love sent to me:
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the eighth day of netmas
my true love sent to me:
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the ninth day of netmas
my true love sent to me:
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the tenth day of netmas
my true love sent to me:
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the eleventh day of netmas
my true love sent to me:
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

On the twelveth day of netmas
my true love sent to me:
12 routers on a stick
11 BGP Peers
10 gigs a flowing
9 fans a cooling
8 packets a flowing
7 OSPF areas
6 switches switching
5 poe injectors
4 subnet masks
3 sfps
2 ethernet ends
and a spanning tree instance

Networking foundations

In an article earlier today, I wrote about certifications and the ISP.  The biggest area I see folks go wrong when it comes to networking is having a good understanding of design. One of the analogies I like to use is building a house.   You have several key roles when you build a house.  These can be directly applied to the networking world.

The first is the Architect. Everything starts with this person or team’s vision. The Architect lays out the design and how the network will function. This person needs a wide variety of skills.  They know the product lines they are supporting, they know how these products fit into the overall vision of the network, and they know the limitations of what they are working with, to name just a few of their many skills. These are your CCIE and higher level CCNP folks in certification terms.  They have enough breadth of knowledge to see the entire picture.  Not just what the device in front of them can do. A network architect would know that a certain wireless CPE does not fully support a VPLS tunnel, and would either recommend not using that equipment or come up with a workaround to implement it into the network, if in fact, they were using VPLS.  Many large companies have Architects who implement and validate network designs. These are then pushed to the next group of folks to implement.

The “tradesmen” or “tradeswomen” are the ones who actually implement the designs around the Architects blueprints for the network.  Just like a house who has carpenters, brick layers, and roofers, so do you have folks who know wireless, routing, security, and other disciplines.  These are the folks who can make the machines do what they want them to do. They work off the blueprints to actually make the network talk and function according to the Architects design. these are your CCNA and CCNP level folks. These folks know the equipment configuration in and out.  They are the most responsible for making things work, and knowing how to make it work. The more experienced of these folks typically collaborate with the Architects to provide expert opinions on the latest features of the equipment they are implementing or any limitations of the equipment.

Many folks working in the ISP world wear multiple hats at the same time.   There is nothing whatsoever wrong with this.  You just have to know the limitations of yourself and the things you have to work with. I see multiple illustrations of this on a daily basis. Clients take a router and make it do BGP, OSPF, PPPoE termination, firewall rules, and other things.  Sometimes this is a budgetary thing, maybe just a lack of understanding, or it can even be a sales hype thing. however, not having an understanding of the design and architecture of a network can be costly.

Anyone can build a house. Go to the lumberyard and get some materials, some tools, and watch a few YouTube videos. Bam you are set.  That will probably work until the first time it rains, or it gets cold. Then you are wondering where your design went wrong.  Using plastic on your roof sounded like a good idea until the wind ripped it.  Same can be said in networks.  Start and always have design considerations in mind.  Not just design, but how individual components are best used in that design. Then rely on the tradesmen  to implement them.  You might be one and the same, but don’t wing it as you go.

Helpful outages web-site for network admins

http://www.outages.org/

Direct from their web-site.

How to Report
By sending a tweet with any of the following hashtag/s:
#outages
#outage
#cablecut
#fibercut
#undersea

when reporting for a service outage. Once verified we will plot it on tracker.

For e.g. #outage #loc (street, city – location name) #start (time), followed by #back (time)#planned or #unplanned (if its a planned or unexpected outage).

Download the iOS/iPhone/iPad App “Ushahidi” and add http://tracker.outages.org/ to “Settings” to start reporting.
Download the Android App “Ushahidi” and add http://tracker.outages.org/ to “Settings” to start reporting.
By sending an email:
outages@outages.org
Click to View Reports

Send comments/feedback/feature requests tovirendra[dot]rode[at]outages.org

Follow us on Twitter @outagesdotorg

Help spread the word!

MTIN WISPA Announcement

MTIN would like to announce some exciting new services for ISPs and network operators

The first is Midwest Internet Exchange ( www.midwest-ix.com )

MidWest-IX has created a peering fabric we are expanding to data centers focused on the needs of WISPs and network operators such as yourself. Peering can be a valuable, and cost-effective solution for your ISP. MidWest-IX has created a solution based around those needs.

We are tailoring and providing these services to the WISP community as a way of making everyone stronger. WISP operators need advantages. MidWest-IX can provide lower latency to content providers such as Netflix. MidWest-IX can cut down on transit costs through peering. We are also creating an ever-growing marketplace for members to provide redundancy, market goods and services to each other, and create a WISP peering cloud. We have many more benefits of an exchange listed at: http://www.midwest-ix.com/benefits.html

We have exchange services available at:

350 East Cermak Chicago Illinois
733 West Henry Street Indianapolis Indiana
401 North Shadeland, Indianapolis Indiana
900 Walnut St. Louis Missouri
535 Scherers Court Columbus, Ohio

If you are a WISP in Indiana, Ohio, Illinois, Kentucky, Michigan, or Missouri contact us on how we can leverage the exchange to help your business. Other locations planned for 2015.

Our next announcement includes services in several Data Centers.

MTIN in cooperation with Midwest Internet Exchange (M-IX) offers co-Location, bandwidth, peering, transport, and managed services.

Do you have a need for circuit termination, server/router space, or peering in any of the above locations? Let us put together a managed solution for you. MTIN can handle the ins and outs of cross-connects, facilitating ports to the exchange fabric, and other data center needs. A data center can be an intimidating thing. Let us take the guesswork out of it for you.

Services Include
-Bandwidth (let our experts provide unique and out-of-the-box solutions)
-Transport
-Cross connects and cable landings
-Off-site backup and DR
-Co-location (TierIV and basic Co-location)
-Connections to 3rd Party networks such as Internet2
-BGP Peering

MTIN provides xISP consulting and backend solutions. BGP, OSPF, routing, DNS, network engineering, and other services. Talk to us how we can put together a complete solution to optimize your network. Our Engineers can design a cost-efficte solution that fits you and your needs.

Contact us:

Web: www.mtin.net
Web: www.midwest-ix.com
e-mail: support@mtin.net
Tel: 317.644.2224

Modular advantages

Several of you have heard me talk about modular network design. There are five distinct advantages of a modular network design.

1.Scalable
You can upgrade pieces easier than all-in-one devices. Being able to upgrade a certain module for that big new client might mean the difference between being able to do it in 10 days instead of 30.

2.Resilient
You can avoid single points of failure. You are not depending on “big iron” for everything. You have multiple modules as a part of your design.

3.Performance
If routers, switches, access points, or whatever are doing less then you get better performance out them. CPUs are dedicating more time toward less processes. This results in better performance. Routers that are doing just BGP are able to process route tables better than ones doing stuff like OSPF or radius.

4.Flexible
You are less dependent on certain vendors, or product families. Instead of having some big router that does BGP, OSPF, and other functions you can have a modular design where a router does, lets say, just BGP. Now you have more choices in hardware because it might not. This allows you to try out newer products or new product features.

5.Easier to maintain
Doing software upgrades on a modular devices, which is doing less things, means bugs and interoperability things don’t happen as often. Every piece of code has bugs. If the device is doing 5 tasks instead of 15 then the chance of a bug affecting the device is less. It also allows you to compartmentalize certain parts of the network.