Some ePMP 550 and sectors.
Some ePMP 550 and sectors.
A discussion which comes up over and over in the WISP space is grounding and proper installation of customer CPE. The folks at perfect-10 (https://www.perfect-10.tv/) were a vendor at #WISPAPALOOZA2018. One of the best things I have seen them in a long time is the below photo they created. This is a great illustration of how a proper CPE goes.
Another WISPAPALOOZA has come and gone to Las Vegas. If you are not familiar with WISPAPALOOZA it is one of the two major trade shows Wispa puts on each year. The Las Vegas show is the larger of the two. If you are a Wireless Internet Service Provider (WISP), doing outdoor wireless networks, or involved in the WISP industry whatsoever this is the premier event for you.
Having attended several shows I have been able to see the evolution of the shows over the past X years. The industry has been evolving and this show highlights many of the evolutions the industry is undergoing. This year billing, LTE, and business services needed by WISPs were prominent in the sessions and on the vendor floors. Vendors such as Baicells, Cambium, Redline, and others were talking about their LTE offerings.
Other vendors such as Ritalia funding and Unitel were there getting their message out to WISPs. As WISPs evolve, they are looking for services and products which help the business side of things. This business side becomes as important in a mature WISP as the technical side is.
Another aspect WISPs look for as they grow are billing platforms. Companies such as Sonar, Azotel, and VISP were represented at the Vegas show. These can help your WISP streamline and automate operations. The more you can stretch your resources the better.
Other themes of the show included fiber deployments, MDUs, and CBRS to name just a few. Knowledge exchange is a key highlight of shows such as this. Operato
rs sitting around trading tips, war stories, and what is working and not working for them is invaluable. The selection of the speakers and sessions highlight the pulse WISPA has on the industry as a whole.
Due to the varied mix of people in this WISP industry shows such as this are great for any level of employee. Any employee will be able to find things they can be engaged with and hopefully add to their knowledge. As with any industry, making personal connections and people networking is huge these events. Being able to have a Rolodex of Vendors, distributors, consultants, and other folks in the industry are invaluable. Whether you forge relationships over a beer at the local bar, a walk down the strip, or just sitting at a table talking these conversations are something you can’t get anywhere else.
My week started with a 5 am arrival at the airport. I was able to catch up with a friend in the industry over a coke at breakfast in the airport. Even though he lives an hour south of me we don’t get together all that much. He is a friend and someone who is doing cool stuff in the industry. The ability to have that focused time is something many of us don’t get. We are so busy with our own lives we have to get into situations like that. After landing in Vegas the intensity level ramps up. You start seeing people you know and quick conversations turn into 20-minute conversations, and the next thing you know its 6 pm and you are just checking into your room. The next several days are more of the same. Late nights, lots of conversations, lots of walking, and an overwhelming amount of information you will only begin to digest once you have a moment to relax. In the end it is a good kind of tired.
For more information and a show re-cap visit http://thebrotherswisp.com/ for a cast coming soon on a re-cap of some of the brothers discussing the show.
One of our climbers took this picture while climbing to do an install for a WISP client inside a “golf ball” water tower.
Due to several opportunities, MTIN is moving toward collapsed support plan style. We will only be offering a contract support model and best effort service.
Customers purchase a minimum 5 hour block for $445. Rates are billed as follows.
Normal Business Hours (8am-5pm EST time) $89 per hour
Late night & Weekend support $125 per hour
Contract customers will use a minimum of 1 hour a month for remote monitoring services, firmware, and other notifications, as well as any general maintenance tasks in order to keep documentation and other aspects of the network up to date for our technicians. Remote Backups, security audits, and other services which MTIN hosts for the customer can be included in the one-hour minimum.
The goal with contract customers is to build a long-term relationship in order to help their networks grow and thrive. By being engaged with the customer on a regular basis we are able to provide better service and have a better handle on customer needs. As networks become more and more complicated having a consultant who has is more a partner than a hired gun is becoming valuable to an operator.
Tier II emergency response times for Tier II customers.
Normal working hours (3 hour maximum)
Late night and weekend (5 hour maximum)
Best Effort Plan
Customers pay as they go for best effort support. Support is available on a best effort service and first come first serve.
Normal Business Hours (8am-5pm EST time) $119 per hour
Late night & Weekend support $145 per hour
1. Late night and Weekend is defined as. 9PM-9AM EST MONDAY-FRIDAY & 7PM EST FRIDAY -9AM Monday. Contract Customers can schedule maintenance windows and such at a discounted rate ahead of time.
2. All customers who don’t have pre-purchased time will be served on a best effort service. Priority will be given to contract customers, and then “walk-in” customers.
3. All times stated are maximum times for response. Depending on workload, times are typically much less.
4. Contact MTIN for details on this new plan.
One way to setup a Baicells POP using VLANs and a VPLS tunnel. The tunnel is to save Public IP space if you so desire.
In 2006 I was hired on part-time for Purdue University. My days would consist of mornings on Campus at Purdue doing I.T. support for the Agronomy Department and afternoons, evenings, and weekends doing support and build-outs for NDWave. We were in the mode of dumping everything back into the company to get to a solid sustainable position. I didn’t really want to go back to desktop support, but the benefits and part-time position helped to pay off bills we had accumulated with MTIN. Purdue allowed me to meet Donnie Payne. Donnie is an infectious personality sort of person. Just being around him motivates you to do things. Purdue allowed me to work with Mac OSX Xserves, cutting-edge Linux servers, and special projects. One of the coolest projects I worked on included remote sensor trailers. We had servers, remote reboots, and had to deal with how to get connectivity out in BFE. Several sites had cellular or satellite uplinks. I was able to apply my ISP knowledge to this project.
NDWave exposed us to lots of rockstars in the ISP industry. JohnnyO and his crew helping with removing feedhorns from a tower is something I will never forget. Sitting outside a hotel with him cooking dinner. Guys like Chuck Hogg, who helped the industry in several ways and is just a plain cool cat. The work alone has opened up friendships which were well worth any long days. Guys like Jay Panozzo, who are not directly connected with the WISP industry, but have their parts. Jay owns Midamerica Towers and is a Man among men. Jay sets the bar for the tower professional.
I continued to dedicate time to Purdue and Ndwave until April 2008. One month before Omnicity took over management of the NDWave network I became a full time employee. Omnicity started out good, but quickly went downhill after a year for me. With all the lawsuits in place that is about all I will say about that.
After being separated form Omnicity I continued to keep my head above water with steady consulting from companies I had helped over the past couple of years. Kenny Johnson at Mooreland ISP and Scott Reed at NewWays were two of my best customers. I truly understood what it was like to be the one the buck stops with. When you have a tower outage at 1AM. the customer does not care except they have no Internet. Early morning climbs to repair Aps in the dead of winter were not uncommon. That is part of what it takes. Being a former owner really helped me connect with what these guys are going through.
Things were slow so I was able to re-group some. I did not mess with much technology except when I had to. I took a step back and concentrated on the activities I enjoyed. I stepped up collecting G.I. Joe figures visited more friends, and generally did non-tech related things. This in itself expanded those I call friends. My mind and soul needed that healing. Then a funny thing happened. I started writing this blog, becoming active on mailing lists again, and generally became interested in the technology again. I started gaining more and more consulting clients and working with more and more networks. Ubiquiti was just starting to come on the scene, Mikrotik was a mature platform, and prices were starting to come down on gear. I remember sitting around 3 years ago thinking I needed to step up what I am doing. It took me up until then to realize I had enough experience and stuff running around in my head to be an expert. I have watched so-called experts screw up even the simplest things.
In the past couple of years, things have really blossomed in the Industry and I have had the pleasure of being a part of some of it. When you look back on this there are certain key points where you take leaps and bounds. Once I was able to step back and further enjoy things that next leap up was taken. This allowed me to open new pathways of thinking too. My confidence soared, my shyness subsided, and life got better. I attribute this to those who have influenced my life, and the experiences we have shared.
Stripping wireless gear off a 120 foot tower in Chicago with Mike Hammett is one of my fondest memories. The work was hard and long, but having the comrade in arms with you to do it make up for it. There is a bond that is formed which last a lifetime. Mike is one of those guys you want to see succeed. I am fortunate to have been in a position on a couple of occasions to help him when it would have been a paid for him to hire it out.
This brings us to present day…
Continued in part 5 (the conclusion I promise)
One night I am sitting in my office wondering what is next for MTIN and I get a call from Steve Narducci in Anderson, Indiana. Steve has this idea he wants to start an ISP. I call up one of my good friends Chris Orr. Chris and I had become good friends out of a chance meeting of him stopping by the office for some thermal Paste. I instantly knew Chris was of the same kind of mold I was. Chris had been hanging out at the office and helping with MTIN for sometime now. Chris is the best *NIX engineer I have ever seen. So I call Chris and ask if he wants to make a little money and so something enjoyable. I think it took Chris awhile to realize I don’t let much hold me back and life is all about going for opportunities or creating ones.
Early one Saturday morning in 2006 Chris, Amber, and I head to Anderson Indiana to hang the first Access Points for what would become ndwave.com. We had been prepping for this for months. T1 line had been ordered, servers built, and equipment ordered and delivered. Little did any of us know we were on the verge of something great. Through a small team we were able to grow to over 1,000 customers under 2 years. Working with NDWave was one of the first times I had the complete package. I had the freedom to shape a growing network and the financial backing to do it. I was as unrestricted as I could get. I felt like I had finally arrived into what I was supposed to be doing. We were working hard and long hours, but it was fun. There is an old saying that goes something like “If you find a profession you truly love, you will never have to work another day in your life”.
During this time I really was able to get to Know Rick Harnish. Rick is the Marlon Brando of the the Wireless ISP world. Rick was eager to share what he knew and help everyone around him grow. Having someone like Rick to have conversations with was a huge asset. He was a major pipeline to the innovation and direction other WISPs were going. Plus Rick is just a plain cool guy.
NDWave really established my credibility in the ISP world. I had been looking for that recognition for quite awhile. Folks like Michael Pelsor, & Debbie Seal would be added to the “family”. These are folks who I consider friends to this day. It was like TCTC all over again. We were on the leading edge of this Wireless ISP wave. The technology was becoming easier to use and more affordable. This meant the average person could now afford reliable service delivered via Wireless. We were growing into areas where there was no broadband. It was kind of like the Wild West gold rush. There were weeks NDWave was putting up 3-4 towers. I was getting to work with Mikrotik, Cisco, Tranzeo, and some other manufacturers. Life was good. We had a fiber feed, rack space at a Premier data center, and got to play with other cool toys.
Then Omnicity comes along and things change yet again….
Recently there have been discussions on some lists about carrier redundancy. I figured I would sum up some thoughts and add my own,
In today’s world of consolidation, takeovers, and cost saving measures carrier redundancy is something one should pursue with due diligence. Below are some questions to know about your existing provider and any future providers. If you know this you can compare the differences between two providers. By knowing the answer to these questions you can add two carriers which will complement each other.
1.Where does my circuit go when it leaves my equipment? Look at this from a regional perspective. Where does it travel in the city? Where does it travel to the next city?
2.Does the provider’s lines share conduit with other providers? They might not know this, but if you have two providers you can compare routes. If they are in the same conduit or in separate conduit in the same trench that might not be ideal. A backhoe could take both out. Do they share space inside the path with other carriers as well? If so, this could cause issues with contract disputes, not paying bills, and other business-related functions. Imagine if carrier A is sharing conduit with Carrier B. Carrier A goes out of business and holds the conduit contract. Where does that leave Carrie rB?
3.Where is the entry point to the facility for the provider’s circuits? If both come into the same part of the building this could be a potential weak point. Ideally one would enter from the north (or south or whatever) and the other would enter from a different direction. Also, they would travel up different conduits on different sides of the building. This way if something like a car crashes through the building may be one of them will be protected.
4.Does the provider farm any of your circuit out to a 3rd party? This is good to know when problems arise and the finger pointing begins.
5.Use tools such as a “Looking Glass” to see if there are differences in routes. If you have two backbone providers and they have very similar routes to reach the major sites (ie. Google, Yahoo, etc.) then you could open yourself up for problems with latency and packet loss should those paths become congested of fail. Ideally, you want ProviderA to have different routes than ProviderB. This way if something outside their network is causing issues it won’t have as big of an impact on your network. Think of this as a road. You might have two roads leaving your town, but you don’t want both of those roads taking the same path to get to the outside world.
Also look at this from your own equipment perspective. If you terminate all your circuits on a single router you are dependent on that router. Same goes for anything. If everything comes in over the same ladder racks that are a point of failure. If all your equipment is in the same room that is a point of failure.
Redundancy can be as diverse as you want to. It boils down to mitigating the risk. If you know all the risks you can say “Yeah I am willing to bring my cross connects over a single ladder rack because the likely-hood of that rack failing is a risk I will take.”
This was an older article I had on my blog a few years ago. Much of this applies still.
Bandwidth is a big hurdle most aspiring WISPs face. The reason is if high-speed alternatives were already in place, the need for a WISP would not be as great. Sure there are business models in which the WISP can compete with other high-speed solutions. However, the bread and butter of a WISP is going into underserved areas.
You have several options for bringing a connection into your area to re-distribute to your customers. I will outline these and then go into further detail
-Leased Lines (Fractional, T-1, T3, etc.)
Leased Lines are the most easily accessible across the United States. However, as more and more providers build fiber it is taking over as the preferred method of connectivity. Fiber is more “future proof” than a T-Carrier circuit such as a T1 or T3. Most phone companies can provide t1 service to almost anywhere. This is because T1 service uses the existing copper already at 99% of locations. If you have a phone line you can almost always get t1 service. Once you go beyond T1 things get a little more complicated. However, T1 has the ability to do bonding if the carrier and telco support it. You essentially buy multiple T1s and combine them into a single “pipe”. This requires the provider to support bonding as well as some special configuration on your routers.
Some questions you should ask your provider/telco.
1.Where is my circuit “homed out of”? This means where does the circuit terminate on the facility end. You do not want this to be too far. If it is too far your reliability will suffer because you have more distance and equipment to go through. This raises the likelihood of an equipment failure, backhoe digging something up, & utility poles falling. The longer the distance also means the “loop charge” will most likely increase. We will get to that in a moment.
2.There are several types of T1s for our purposes. Some terms to familiarize oneself with are PRI, channelized, transport, and port fee.
3. Ask your provider to spell out what type of t1 this is. If you are buying the T1 from a backbone provider such as Qwest, Level3, and others they will typically bundle everything into one package. Ask them to break this down if they don’t. You want to know what the Local loop charge is, what the port fee is, and what the bandwidth costs. The local loop is typically what the telephone company charges to deliver the circuit from Point A (their equipment) to Point B (you). If you are going with a 3rd party, and not the local telephone company, the provider typically becomes the central point of contact for the entire circuit. This can add a level of complexity when issues arise.
The port fee is a charge normally passed on for connecting to the provider’s equipment. Say you have a 48 port switch sitting in a CO-Location facility. For each Ethernet cable you plug in from the telephone company they charge a fee either one-time and/or monthly. This is just the way it is typically. One of those “Because they can” charges. The 3rd charge is the cost of the Internet bandwidth. A T1 can handle 1.5 Megabits of bandwidth so the cost per Megabit is not as big of an issue because you are not buying in bulk.
4.Ask to see the Service Level Agreement (SLA). If you are unfamiliar with the terms have a consultant look this over.
5.Know where your DMARC location is. This is the spot where the provider’s responsibility ends and yours begins.
6.Ask if the provider can verify with the telco how long the next circuit would take to install. You don’t want to go to order a second circuit and find out the local telephone equipment does not have enough capacity. This has happened to our clients on many occasions. This can be a quick process or the telco can take months and months to get around to installing the needed equipment.