WISP LTE, PIM testing, and quality

One of the topics that came up during the Baicells troubleshooting tips was the notion of PIM testing, and cables which are PIM rated.

PIM sweeps are a common thing in the Cellular field.   One of the first questions folks often ask is what is a PIM sweep? If you think of PIM testing as a passive test and line sweeping as an active test that is a good start.  PIM testing looks for problems with things like connectors, cables, and other “layer 1” items.  A PIM test is not a line sweep. Line sweeping measures the signal losses and reflections of the transmission system. this is typically VSWR.  A line sweep is an active test. It can not detect the same things a PIM test can.  Many HAM radio folks are familiar with a line sweep where the reflected power is measure in an antenna system. In a line sweep you deal with reflected power and all that.

What does a PIM test do?

When you do a PIM test typical two high power signals are injected into the antenna line.  You can actually pass a sweep test but not a PIM test.

I won’t go into PIM tests very much because you need high dollar units such as those from Anritsu and Kaelus. These cost 10’s of thousands of dollars new.  Sometimes you can find these used.  However, the next thing you will run into is understanding the output of such a device.  Cell crews go to week long certification classes to become a PIM certified tech from Anritsu and others.

What causes a PIM test to fail?

According to Kaelus the most common problems are:

• Contaminated surfaces or contacts due to dirt, dust, moisture or oxidation.
• Loose mechanical junctions due to inadequate torque, poor alignment or poorly prepared contact surfaces.
• Loose mechanical junctions caused transportation shock or vibration .
• Metal flakes or shavings inside RF connections.
• Poorly prepared RF connections
•Trapped dielectric materials (adhesives, foam, etc.)
•Cracks or distortions at the end of the outer conductor of coaxial cables caused by over tightening the back nut during installation.
• Solid inner conductors distorted in the preparation process causing these to be out of round or tapered over the mating length.
• Hollow inner conductors excessively enlarged or made oval during the preparation process.

Why does cable matter?

Cables do not typically cause PIM, but poorly terminated or damaged cables can and do cause problems.

Cables with Seams can cause issues.  The seam can corrode.  Plated copper, found in cheaper cables, can break away from the aluminum core. This actually allows small amounts of flaking to happen between the connector and the core of the cable.  This will cause PIM issues and is very hard to diagnose. Imagine little flakes inside a connector. You don’t see them until you break open the connector, and even then they may be pretty little flakes.

Cables can change their physical configuration as temperature varies. For instance, sunshine can warm cables, changing their electrical length. A cable that happens to be the right length to cancel out PIM when cool may show strong PIM after changing its length on a warm day, or, it can work the other way around, good when hot and bad when cold. In addition, the physical change in length can make a formerly good connection into a poor one, also generating PIM. Other environmental factors such as water in the connector or cable can be an issue, as with any RF setup.

I think I have PIM issues. What are some indications?

PIM often shows up as poor statistics from the affected antenna. One of the first and most direct indications of PIM can be seen in cells with two receive paths. If the noise floor is not equal between the two paths, the cause is likely PIM generated inside the noisy receive path.

How Do I prevent PIM issues?

Cable quality and connector quality are one of the biggest factors in the PIM quality of a LTE system.  Many WISPs are used to making their own LMR cables and putting on their own connectors.  There is a difference between a low PIM LMR-400 cable and normal LMR-400.  Same for connectors.  One of the recommendations today was to use 1/2” superflex heliax.

The easy recommendation is to buy pre-made cables that have already been PIM certified.  In a typical WISP setup, you do not have lots and lot of components in your setup. Buy already certified components from your distributors that are “Low PIM rated”.

Libre and Cambium

What Can Libre do for the Cambium operator?

Number of registered SMs. This AP has 7

 

Interface traffic

Number of GPS Satellites tracked

Modulation

Libre can be setup for thresholds.  Any of the above graphs can be set up to alert based upon a threshold.  You could setup an alert that says if SMs on an AP drop by 10% kick out an alert.  This could alert you to possible frequency issues.   You could set a threshold for if traffic reaches a certain level.

Learning, certifications and the xISP

One of the most asked questions which comes up in the xISP world is “How do I learn this stuff?”.   Depending on who you ask this could be a lengthy answer or a simple one sentence answer.  Before we answer the question, let’s dive into why the answer is complicated.

In many enterprise environments, there is usually pretty standard deployment of networking hardware.  Typically this is from a certain vendor.  There are many factors involved. in why this is.  The first is total Cost of Ownership (TCO).  It almost always costs less to support one product than to support multiples.  Things like staff training are usually a big factor.  If you are running Cisco it’s cheaper to train and keep updated on just Cisco rather than Cisco and another vendor.

Another factor involved is economies of scale.  Buying all your gear from a certain vendor allows you to leverage buying power. Quantity discounts in other words.  You can commit to buying product over time or all at once.

So, to answer this question in simple terms.  If your network runs Mikrotik, go to a Mikrotik training course.  If you run Ubiquiti go to a Ubiquiti training class.

Now that the simple question has been answered, let’s move on to the complicated, and typically the real world answer and scenario.  Many of our xISP clients have gear from several vendors deployed.  They may have several different kinds of Wireless systems, a switch solution, a router solution, and different pieces in-between.  So where does a person start?

We recommend the following path. You can tweak this a little based on your learning style, skill level, and the gear you want to learn.

1.Start with the Cisco Certified Network Associate (CCNA) certification in Routing and Switching (R&S).  There are a ton of ways to study for this certification.   There are Bootcamps (not a huge fan of these for learning), iPhone and Android Apps (again these are more focused on getting the cert), online, books, and even youtube videos. Through the process of studying for this certification, you will learn many things which will carry over to any vendor.  Things like subnetting, differences between broadcast and collision domains, and even some IPV6 in the newest tracks.  During the course of studying you will learn, and then reinforce that through practice tests and such.  Don’t necessarily focus on the goal of passing the test, focus on the content of the material.  I used to work with a guy who went into every test with the goal of passing at 100%.  This meant he had to know the material. CompTIA is a side path to the Cisco CCNA.  For reasons explained later, COMPTIA Network+ doesn’t necessarily work into my plan, especially when it comes to #3. I would recommend COMPTIA if you have never taken a certification test before.

2.Once you have the CCNA under your belt, take a course in a vendor you will be working the most with.  At the end of this article, I am going to add links to some of the popular vendor certifications and then 3rd party folks who teach classes. One of the advantages of a 3rd party teacher is they are able to apply this to your real world needs. If you are running Mikrotik, take a class in that. Let the certification be a by-product of that class.

3.Once you have completed #1 and #2 under your belt go back to Cisco for their Cisco Certifed Design Associate (CCDA). This is a very crucial step those on a learning path overlook.  Think of your networking knowledge as your end goal is to be able to build a house.  Steps one and two have given you general knowledge, you can now use tools, do some basic configuration.  But you can’t build a house without knowing what is involved in designing foundations,  what materials you need to use, how to compact the soil, etc.  Network design is no different. These are not things you can read in a manual on how to use the tool.  They also are not tool specific.   Some of the things in the Cisco CCDA will be specific to Cisco, but overall it is a general learning track.  Just follow my philosophy in relationship to #1. Focus on the material.

Once you have all of this under your belt look into pulling in pieces of other knowledge. Understanding what is going on is a key to your success.  If you understand what goes on with an IP packet, learning tools like Wireshark will be easier.  As you progress let things grow organically from this point.  Adding equipment in from a Vendor? Update your knowledge or press the new vendor for training options.  Branch out into some other areas ,such as security, to add to your overall understanding.

Never stop learning! Visit our online store for links to recommend books and products.

WISP Based Traning Folks.
These companies and individuals provide WISP based training. Some of it is vendor focused. Some are not.  My advice is to ask questions. See if they are a fit for what your goals are.
-Connectivity Engineer
Butch Evans
Dennis Burgess
Rickey Frey
Steve Discher
Baltic Networks

Vendor Certification Pages
Ubiquiti
Mikrotik
Cisco
Juniper
CWNA
CompTIA

If you provide training let me know and I will add you to this list.

Simple shut-off scripting

I had a client today who is doing some manual things as they are using Quickbooks for billing and such.  One thing they kind of struggle with is turning off people for non-payment and such.  Their current method is adding a que and throttling someone to a low-speed to make them call.  Their network is a routed network utilizing DHCP to the CPE at the customer.  Everything is in router mode and they control the addressing of the units via DHCP reservations.  So how do we make this better without adding radius and all kinds of stuff into the network?

First we set up a web-proxy

/ip proxy
set enabled=yes port=8089

/ip proxy access
add dst-host=mtin.net dst-port=80
add dst-host=*.mtin.net dst-port=80
add dst-port=53
add action=deny redirect-to=www.mtin.net

What the above code does is says anyone coming into the proxy is only allowed to go to mtin.net (used our domain as an example), use port 53 (DNS), and anything else gets redirected to www.mtin.net. We chose port 53 because they are in the process of cleaning up some of the radios and such which are using 8.8.8.8 and other DNS servers.

Next we set up a nat rule

/ip firewall nat
add action=redirect chain=dstnat dst-port=80 protocol=tcp src-address-list=\
SHUTOFF to-ports=8089

This nat rule says anyone making a port 80 request coming from our SHUTOFF address-list gets redirected to port 8089 (our proxy port setup earlier).

Our third step is to setup our address list. this is very straightforward.  Just modify and add users to this list when they are to be turned off.

/ip firewall address-list
add address=10.20.0.192 list=SHUTOFF

Lastly, we add a filter rule which denies the SHUTOFF folks from using anything except port 53 and port 80.  We do this because we can’t proxy port 443 and other SSL traffic. If folks go to a HTTPS site it simply fails.  This is a drawback of using a web-proxy.

/ip firewall filter
add action=drop chain=forward dst-port=!53,80 protocol=tcp src-address-list=\
SHUTOFF

If you have an SSL payment gateway you can modify your filter rules to allow traffic to it. This is just one quick and dirty way of letting customers know they have been turned off.

Homeland Security US-Cert e-mail on Network infrastructure

A few days ago Homeland Security published an e-mail on threats to network devices and securing them.  Rather than cut and paste I exported the e-mail to a PDF. Some good best practices in here.

TA16250A The Increasing Threat to Network Infrastructure Devices and Recommended Mitigations

Leap second to be added in 2016

http://www.space.com/33361-leap-second-2016-atomic-clocks.html

A “leap second” will be added to the world’s official clocks on Dec. 31 at 23 hours, 59 minutes and 59 seconds Coordinated Universal Time (UTC), which corresponds to 6:59:59 p.m. EST; the clocks will read 23:59:60 before ticking over to midnight. The goal is to keep two different timescales in sync with each other.

So, why is this important to you as an ISP?
The trouble is that even as they use the leap second, UNIX and Linux define a day as something that is unvarying in length. “If a leap second happens, the operating system must somehow prevent the applications from knowing that it’s going on while still handling all the business of an operating system,” says Steve Allen, a programmer with California’s Lick Observatory.
Source:http://www.wired.com/2015/01/leap-second-rattle-internet-theres-plot-kill/

Many patches and fixes have been put in place to adjust for this in most Operating systems.  The concern, even a year after the last one, is software that has not been updated to account for dealing with leap seconds.

Mimosa G2 first look

So Scott @ On-Ramp Indiana ordered a few of these and figured I would borrow one for a few days and do a first look and review on them.

Mimosa G2 Box

Nice and compact box.  I am a fan of the cover.  If this way on a store shelf I would notice.

G2 Insert
The very first thing you see is this wireless information card. Very handy for the home user. Many of my clients throughout the years save such things so having this in a bright card is a nice touch.  Another nice feature of this card is it has sticky tape on the back.  You can actually peel it off and stick it somewhere.  Not everyone has a network rack, so affixing it to there might not be the best bet.  We are in the day and age where there are not “telephone stations”.  The only thing I could come up with might be in a desk drawer or something like that.  I could see guests asking for the key so you would want this handy.  Any thoughts on a good place to put this?

Information Card
On the back of the card is a very handy diagram on the 3 modes of this device.  You can use it as a Wifi router. You can also use it as a repeater.  In this mode it works both wired and wireless.  As with some other manufactures it will auto-configure itself to join in with the rest of the network.  It learns the configuration and away it goes.  Thirdly, is a simple pass through mode. This is helpful if there is another router involved.

G2 contents
The box contained the unit, a slip on power plug, and the compliance paperwork no one reads. Thank you Lawyers.  One of the first things I noticed about this unit is the well made feel to it.  The plastic does not feel cheap, and it feels heavy.  That is always a scientific measure right?

G2 Ports
Mimosa has done a good job of helping the uneducated user on the use of this product.  A good example of this is plug, which is in the POE port.  This plug takes a little bit of effort to remove.  As you can see in the picture, it is also marked with a red label to distinguish this from the customer side.  This is so the customer doesn’t feed 48 volt to their router, laptop or whatever gets plugged in.

IMG_3553
Also, you have holes on the top and bottom for cooling.  On the side is a very easy to get to reset button. Another nod to Mimosa paying attention to common issues home users run into is there is a very clear sticker on the top of the unit which has the Home network SSID and passphrase on it.  A user can simply walk to the unit, look down and easily read the needed information.

G2 Plug

The power plug simply slides in a groove and snaps in place.  Nice clean setup.

In closing, my first impressions of this product are positive.  Packaging and instructions are put together well and easy to understand. The product feels good and has a good number of things to address common issues. Look for part two of this for a look of how this actually works, configuration, and testing.

If you are a manufacturer and have a product you think we would be interested in reviewing please contact us.