SaaS aka why I should pay per month for billing

The topic of paying per user for a billing or management platforms comes up every so often.  I was able to sit down and talk with several vendors at WISPAPALOOZA this year about the value of their customers paying a per-user fee.

The most prevalent thought is about innovation and new features.  SaaS allows the billing vendor to invest development and testing time in rolling out new features to support new equipment, and other software.  LTE platforms are the hot thing in billing integration. New additions to software take people power and hours of testing and tweaking. Without monthly recurring revenue to drive such things billing vendors would have to develop this and then charge to the early adopters as an add-on.  This can be a double-edged sword. The early adopters have to pay a premium in order to get a partial solution because the vendor has to really prioritize how their development resources are used. The Vendor is always chasing the next big thing, which means other additions or fixes tend to get pushed back. They have to finish add-ons they think more folks will want to buy first.

The next thing is plain old hosting. Hosting a software application, whether in the cloud or on your own hardware costs money.  Co-location, software patches on the OS, hardware lifecycles, etc.  This cuts down on the end-user maintenance side of the hardware but pushes it back to the vendor. The peace of mind of knowing the thing that collects your money is running is backed up, and is available as part of the monthly fee you pay.

SaaS also allows for quicker releases of bugs and new features.  Vendors have more resources dedicated to development and changes. This allows for new add-ons to become available quicker.  Take the traditional model where you get bug fixes, but major feature add-ons are either a full point upgrade or major version upgrade. This usually costs money and is a slower process.  Not only does the vendor have to spend resources advertising, but they have to deal with support and other issues. With billing vendors who charge a monthly fee fixes from companies such as Paypal or Authorize.net are almost always rolled out very quickly at no additional charge to the end user ISP.

Some companies such as Basecamp, which is not a billing platform, have taken a hybrid approach to SaaS. Every major revision that comes out is an upgrade. You can choose to upgrade or stay where you are and pay the same amount.  This can leave customers behind but still allows them to use what they are paying for.  They just don’t get new features or bug fixes.

So the next time you are figuring out why you should pay for a billing platform on a monthly, customer, or subscription basis take all of this into account.

For those looking for xISP billing, and mainly WISP billing, here is a partial list:

www.azotel.com
www.visp.net
www.powercode.com
www.sonar.software
www.splynx.com
www.ispbilling.com (Platypus)
www.freeside.biz
www.quickbooks.com

If you have more please add them in the comments.

The problem with peering from a logistics standpoint

Many ISPs run into this problem as part of their growing pains.  This scenario usually starts happening with their third or 4th peer.

Scenario.  ISP grows beyond the single connection they have.  This can be 10 meg, 100 meg, gig or whatever.  They start out looking for redundancy. The ISP brings in a second provider, usually at around the same bandwidth level.  This way the network has two pretty equal paths to go out.

A unique problem usually develops as the network grows to the point of peaking the capacity of both of these connections.  The ISP has to make a decision. Do they increase the capacity to just one provider? Most don’t have the budget to increase capacities to both providers. Now, if you increase one you are favouring one provider over another until the budget allows you to increase capacity on both. You are essentially in a state where you have to favor one provider in order to keep up capacity.  If you fail over to the smaller pipe things could be just as bad as being down.

This is where many ISPs learn the hard way that BGP is not load balancing. But what about padding, communities, local-pref, and all that jazz? We will get to that.  In the meantime, our ISP may have the opportunity to get to an Internet Exchange (IX) and offload things like streaming traffic.  Traffic returns to a little more balance because you essentially have a 3rd provider with the IX connection. But, they growing pains don’t stop there.

As ISP’s, especially WISPs, have more and more resources to deal with cutting down latency they start seeking out better-peered networks.  The next growing pain that becomes apparent is the networks with lots of high-end peers tend to charge more money.  In order for the ISP to buy bandwidth they usually have to do it in smaller quantities from these types of providers. This introduces the probably of a mismatched pipe size again with a twist. The twist is the more, and better peers a network has the more traffic is going to want to travel to that peer. So, the more expensive peer, which you are probably buying less of, now wants to handle more of your traffic.

So, the network geeks will bring up things like padding, communities, local-pref, and all the tricks BGP has.  But, at the end of the day, BGP is not load balancing.  You can *influence* traffic, but BGP does not allow you to say “I want 100 megs of traffic here, and 500 megs here.”  Keep in mind BGP deals with traffic to and from IP blocks, not the traffic itself.

So, how does the ISP solve this? Knowing about your upstream peers is the first thing.  BGP looking glasses, peer reports such as those from Hurricane Electric, and general news help keep you on top of things.  Things such as new peering points, acquisitions, and new data centers can influence an ISPs traffic.  If your equipment supports things such as netflow, sflow, and other tools you can begin to build a picture of your traffic and what ASNs it is going to. This is your first major step. Get tools to know what ASNs the traffic is going to   You can then take this data, and look at how your own peers are connected with these ASNs.  You will start to see things like provider A is poorly peered with ASN 2906.

Once you know who your peers are and have a good feel on their peering then you can influence your traffic.  If you know you don’t want to send traffic destined for ASN 2906 in or out provider A you can then start to implement AS padding and all the tricks we mentioned before.  But, you need the greater picture before you can do that.

One last note. Peering is dynamic.  You have to keep on top of the ecosystem as a whole.

WPA is not encrypting your customer traffic

There was a Facebook discussion that popped up tonight about how a WISP answers the question “Is your network secure?” There were many good answers and the notion of WEP vs WPA was brought up.

In today’s society, you need end-to-end encryption for data to be secure. An ISP has no control over where the customer traffic is going. Thus, by default, the ISP has no control over customer traffic being secure.  “But Justin, I run WPA on all my aps and backhauls, so my network is secure.”  Again, think about end-to-end connectivity. Every one of your access points can be encrypted, and every one of your backhauls can be encrypted, but what happens when an attacker breaks into your wiring closet and installs a sniffer on a router or switch port?What most people forget is that WPA key encryption is only going on between the router/ap and the user device.  “But I lock down all my ports.” you say.  Okay, what about your upstream? Who is to say your upstream provider doesn’t have a port mirror running that dumps all your customer traffic somewhere.  “Okay, I will just run encrypted tunnels across my entire network!. Ha! let’s see you tear down that argument!”. Again, what happens when it leaves your network?  The encryption stops at the endpoint, which is the edge of your network.

Another thing everyone hears about is hotspots. Every so often the news runs a fear piece on unsecured hotspots.  This is the same concept.  If you connect to an unsecured hotspot, it is not much different than connecting to a hotspot where the WPA2 key is on a sign behind the cashier at the local coffee shop. The only difference is the “hacker” has an easier time grabbing any unsecured traffic you are sending. Notice I said unsecured.  If you are using SSL to connect to a bank site that session is sent over an encrypted session.  No sniffing going on there.  If you have an encrypted VPN the possibility of traffic being sniffed is next to none. I say next to none because certain types of VPNs are more secure than others. Does that mean the ISP providing the Internet to feed that hotspot is insecure? There is no feasible way for the ISP to provide end to end security of user traffic on the open Internet.

These arguments are why things like SSL and VPNs exist. Google Chrome is now expecting all websites to be SSL enabled to be marked as secure. VPNs can ensure end-to-end security, but only between two points.  Eventually, you will have to leave the safety and venture out into the wild west of the internet.  Things like Intranets exist so users can have access to information but still be protected. Even most of that is over encrypted SSL these days so someone can’t install a sniffer in the basement.

So what is a WISP supposed to say about security? The WISP is no more secure than any other ISP, nor are then any less secure.  The real security comes from the customer. Things like making sure their devices are up-to-date on security patches.  This includes the often forgotten router. Things like secure passwords, paying attention to browser warnings, e-mail awareness, and other things are where the real user security lies. VPN connections to work. Using SSL ports on e-mail. Using SSH and Secure RDP for network admins. Firewalls can help, but they don’t encrypt the traffic. Does all traffic need encrypted? no.

MTIN is growing again

Over the years MTIN has gone from being a computer repair shop to a dial-up ISP, to a Wireless ISP, and many things in-between.  Each time technology and market conditions change we adapt to change with it.  Our next metamorphosis is needed so we can grow into more aspects of the xISP world In order to accomplish this we are splitting into divisions of what we do.

The first is j2sw.com. This part of the business will be focused on personalised WISP services and support.  These will be custom tailored to a limited number of clients.  Projects such as the “Start a WISP” book and upcoming WISP publications will be run under j2sw.com. Other projects that benefit the ISP community will run from j2sw.com. Having j2sw Consulting as a separate arm allows for better personal attention to key customers.

The second division of the business is MTIN.NET.  This arm will be focused on business to business services such as data center co-location, network connectivity, tower services, and related type services. MTIN is becoming a project management company. We will leverage our vast partnerships to leverage the strength of many to accomplish your medium to large projects.   MTIN will be an umbrella company to bring in the right people for the right projects.

Look for changes to the websites and contact information coming over the next month or so. Justin will be involved with each entity on a very regular basis, but having extra folks can allow for time to be dedicated to ever-expanding projects without sacrificing service to the client.

Some FAQs
Why the change?
For a couple of reasons. The first is to leverage Justin being known in the xISP community.  having a face to the consulting side. This allows for better personal service as well as a trusted name in the WISP community. Secondly, is to allow a better division of resources based on projects and individual needs.

Is MTIN going away?
No, MTIN will move into a project management type of company.  We have access to a large network of contractors, partners, service providers, and other groups we have built since 1998. MTIN can bring in needed resources for projects under one contact point. This allows for projects to not depend on just one person.

Will contact info change?
In the upcoming months, we will be publishing updated contact info. The old information will not go away, but things will get routed to the proper folks better.

For now check out http://j2sw.com and like jswconsulting on facebook.

MTIN announces the support crate plan

Are you a WISP who needs just a little help now and then? Need a sanity check on configuration changes? Need someone who knows your network enough to say whether you need that most recent software upgrade?  Don’t have a big budget for the occasional issue? Need peace of mind you can call someone who won’t break the bank on a simple question? MTIN has a solution for you.

We are calling this the “Supply Drop Plan”. it’s designed for the WISP who needs someone who knows their network and their business for occasional questions outside of their comfort zone.  It consists of the following:
-2 Hours of consulting time a month.
-Reasonable amount of e-mail questions
-Be put on our e-mail notification list of relevant information
$89 a month.

Details
-Access to MTIN via phone during business hours or pre-arranged time (24 hour notice).
-e-mail questions tracked via a ticket system with a maximum of 24 hour response.  Most of the time same day.
-1 year contract

Just some things you can do with your two hours
-Have our engineers look at any new configurations you want to implement
-Unbiased advice on what equipment to order
-Help source equipment for wireless deployments on towers
-Make recommendations on upgrades
-Do audits on things like upstream providers, etc.

What’s not included
-Emergency support (we have plans for that). Emergency support is available but at non-contract rates on a first come first serve basis.
-Additional hours can be purchased on an as-needed basis.  Please note without an hourly block you will be first come first serve.
-Phone calls after hours must be pre-arranged. We can accommodate your schedule. Otherwise, support will be billed at after hours rates.

DMCA Designated Agent Directory updates

The following text is directly from: https://www.copyright.gov/dmca-directory/ 

A relevant F.A.Q. can be found at https://www.copyright.gov/dmca-directory/faq.html

Service Provider Designation of Agent to Receive Notifications of Claimed Infringement

The Digital Millennium Copyright Act (“DMCA”) provides safe harbors from copyright infringement liability for online service providers. In order to qualify for safe harbor protection, certain kinds of service providers—for example, those that allow users to post or store material on their systems, and search engines, directories, and other information location tools— must designate an agent to receive notifications of claimed copyright infringement. To designate an agent, a service provider must do two things: (1) make certain contact information for the agent available to the public on its website; and (2) provide the same information to the Copyright Office, which maintains a centralized online directory of designated agent contact information for public use. The service provider must also ensure that this information is up to date.

In December 2016, the Office introduced an online registration system and electronically generated directory to replace the Office’s old paper-based system and directory. Accordingly, the Office no longer accepts paper designations. To designate an agent, a service provider must register with and use the Office’s online system.

Transition period: Any service provider that has designated an agent with the Office prior to December 1, 2016, in order to maintain an active designation with the Office, must submit a new designation electronically using the online registration system by December 31, 2017. Any designation not made through the online registration system will expire and become invalid after December 31, 2017. Until then, the Copyright Office will maintain two directories of designated agents: the directory consisting of paper designations made pursuant to the Office’s prior interim regulations which were in effect between November 3, 1998 and November 30, 2016 (the “old directory”), and the directory consisting of designations made electronically through the online registration system (the “new directory”). During the transition period, a compliant designation in either the old directory or the new directory will satisfy the service provider’s obligation under section 512(c)(2) to designate an agent with the Copyright Office. During the transition period, to search for a service provider’s most up-to-date designation, begin by using the new directory. The old directory should only be consulted if a service provider has not yet designated an agent in the new directory.

Vendors and core business

I had a client learn a lesson they should not have had to this evening.  The client has had several key servers hosted at a small data center for several years now. These were managed servers the data center took care of. Things like new hard drives were the responsibility of the data center so the client rarely paid attention to these machines.  As many of you know a server can spin for years and it is just forgotten about.

Tonight these servers come under a very heavy Denial of Service (DDoS) attack.  Fifteen plus Gigs come to bear at client’s servers for an extended time.  The client is unable to reach the data center NOC, nor do any of his contacts work.   The servers are knocked offline.  4 hours later the client finally receives an e-mail from the data center saying they unplugged the client’s router because it was taking down their (the DC’s) own network.  After asking to have a call from a manager client finds out the DC has restructured and dropped many of their co-location and other hosting services.  Their multiple 10 gig pipes have been reduced to one, and many clients have left.  The manager says they have re-focused their business to focus on things such as OLED screens, and other things totally unrelated to running a data center. The hosting they do have left “pays the bills” so they can have a place to do research.

The client has redundancy so they are not dead in the water.  However, this redundancy was only supposed to be for a short term duration due to costs.  The lesson learned is to keep in contact with your vital members.  Call up your sales person once or twice a year and see how things are going.  Keep in contact with key folks at the company.  If they are on LinkedIn add the company.  If their focus appears to change or they go silent do some leg work to find out what’s going on.

The Importance of cable support in LTE deployments

As the number of WISP LTE deployments increase, there are many things WISPs will need to be mindful of.  One such item is properly supporting antenna cables. LTE systems are more sensitive to cable issues.  In a previous blog post, I talked about pim and low-pim cables.   One of the things that can cause low pim is improperly mated cables.  If cables are not supported they can become loose over time.  Vibration from equipment or even the wind can loosen connections.

How do we support cables?
We can take a cue from the cellular industry. The following are some examples of proper cable support.  Thanks to Joshua Powell for these pics.

Where can you get these?
A good place to start are sites like sitepro1 or Tessco has a selection.

So the next time you are planning your LTE deployment think about cable support.

Learning, certifications and the xISP

One of the most asked questions which comes up in the xISP world is “How do I learn this stuff?”.   Depending on who you ask this could be a lengthy answer or a simple one sentence answer.  Before we answer the question, let’s dive into why the answer is complicated.

In many enterprise environments, there is usually pretty standard deployment of networking hardware.  Typically this is from a certain vendor.  There are many factors involved. in why this is.  The first is total Cost of Ownership (TCO).  It almost always costs less to support one product than to support multiples.  Things like staff training are usually a big factor.  If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.

Another factor involved is economies of scale.  Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words.  You can commit to buying product over time or all at once.

So, to answer this question in simple terms.  If your network runs Mikrotik, go to a Mikrotik training course.  If you run Ubiquiti go to a Ubiquiti training class.

Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario.  Many of our xISP clients have gear from several vendors deployed.  They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between.  So where does a person start?

We recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.

1.Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S).  There are a ton of ways to study for this certification.   There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of studying for this certification, you will learn many things which will carry over to any vendor.  Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks.  During the course of studying you will learn, and then reinforce that through practice tests and such.  Don’t necessarily focus on the goal of passing the test, focus on the content of the material.  I used to work with a guy who went into every test with the goal of passing at 100%.  This meant he had to know the material. CompTIA is a side path to the Cisco CCNA.  For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.

2.Once you have the CCNA under your belt, take a course in a vendor you will be working the most with.  At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.

3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook.  Think of your networking knowledge as your end goal is to be able to build a house.  Steps one and two have given you general knowledge, you can now use tools, do some basic configuration.  But you can’t build a house without knowing what is involved in designing foundations,  what materials you need to use, how to compact the soil, etc.  Network design is no different. These are not things you can read in a manual on how to use the tool.  They also are not tool specific.   Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track.  Just follow my philosophy in relationship to #1. Focus on the material.

Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is a key to your success.  If you understand what goes on with an IP packet, learning tools like Wireshark will be easier.  As you progress let things grow organically from this point.  Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options.  Branch out into some other areas ,such as security, to add to your overall understanding.

Never stop learning! Visit our online store for links to recommend books and products.

WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not.  My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
Butch Evans
Dennis Burgess
Rickey Frey
Steve Discher
Baltic Networks

Vendor Certification Pages
Ubiquiti
Mikrotik
Cisco
Juniper
CWNA
CompTIA

If you provide training let me know and I will add you to this list.

How I learned to love BGP communities, and so can you

BGP communities can be a powerful, but almost mystical thing.  If you aren’t familiar with communities start here at Wikipedia.  For the purpose of part one of this article we will talk about communities and how they can be utilized for traffic coming into your network. Part two of this article will talk about applying what you have classified to your peers.

So let’s jump into it.  Let’s start with XYZ ISP. They have the following BGP peers:

-Peer one is Typhoon Electric.  XYZ ISP buys an internet connection from Typhoon.
-Peer two is Basement3. XYZ ISP also buy an internet connection from Basement3
-Peer three is Mauler Automotive. XYZ ISP sells internet to Mauler Automotive.
-Peer four is HopOffACloud web hosting.  XYZ ISP and HopOffACloud are in the data center and have determined they exchange enough traffic amongst their ASN’s to justify a dedicated connection between them.
-Peer five is the local Internet exchange (IX) in the data center.

So now that we know who our peers are, we need to assign some communities and classify who goes in what community.  The Thing to keep in mind here, is communities are something you come up with. There are common numbers people use for communities, but there is no rule on what you have to number your communities as. So before we proceed we will need to also know what our own ASN is.  For XYZ we will say they were assigned AS64512. For those of you who are familiar with BGP, you will see this is a private ASN.  I just used this to lessen any confusion.  If you are following along at home replace 65412 with your own ASN.

So we will create four communities .

64512:100 = transit
64512:200 = peers
64512:300 = customers
64512:400 = my routes

Where did we create these? For now on paper.

So let’s break down each of these and how they apply to XYZ network. If you need some help with the terminology see this previous post.
64512:100 – Transit
Transit will apply to Typhoon Electric and Basement3.  These are companies you are buying internet transit from.

64512:200 – Peers
Peers apply to HopOffACloud and the IX. These are folks you are just exchanging your own and your customer’s routes with.

64512:300 – Customers
This applies to Mauler Automotive.  This is a customer buying Internet from you. They transit your network to get to the Internet.

64512:200 – Local
This applies to your own prefixes.  These are routes within your own network or this particular ASN.

Our next step is to take the incoming traffic and classify into one of these communities. Once we have it classified we can do stuff with it.

If we wanted to classify the Typhoon Electric traffic we would do the following in Mikrotik land:

/routing filter
add action=passthrough chain=TYPHOON-IN prefix=0.0.0.0/0 prefix-length=0-32 set-bgp-communities=64512:100 comment="Tag incoming prefixes with :100"

This would go at the top of your filter chain for the Typhoon Electric peer.  This simply applies 64512:100 to the prefixes learned from Typhoon.

In Cisco Land our configuration would look like this:

route-map Typhoon-in permit 20  
match ip address 102  
set community 64512:100

The above Cisco configuration creates a route map, matches a pre-existing access list named 102, and applies community 64512:100 to prefixes learned.

For Juniper you can add the following command to an incoming peer in policy-options:

set community Typhoon-in members 64512:100

Similar to the others you are applying this community to a policy.

So what have we done so far, we have taken the received prefixes from Typhoon Electric and applied community 64512:100 to it.  This simply puts a classifier on all traffic from that peer. We could modify the above example to classify traffic from our other peers based upon what community we want them tagged as.

In our next segment we will learn what we can do with these communities.