The problem with speedtests

Imagine this scenario. Outside your house, the most awesome super highway has been built.  It has a speed limit of 120 Mile Per Hour.  You calculate at those speeds you can get to and from work 20 minutes earlier. Life is good.  Monday morning comes, you hop in your Nissan GT-R, put on some new leather driving gloves, and crank up some good driving music.  Your pull onto the dedicated on-ramp from your house and are quickly cruising at 120 Miles an hour. You make it into work before most anyone else. Life is good.  

Near the end of the week, you notice more and more of your neighbours and co-workers using this new highway.  Things are still fast, but you can’t get up to speed to work like you could earlier in the week.  As you ponder why you notice you are coming up on the off-ramp to your work.  Traffic is backed up. Everyone is trying to get to the same place.  As you are waiting in the line to get off the super highway, you notice folks passing you by going on down the road at high rates of speed.  You surmise your off-ramp must be congested because it is getting used more now.

Speedtest servers work the same way. A speedtest server is a destination on the information super-highway. Man, there is an oldie term.  To understand how speedtest servers work we need a quick understanding of how the Internet works.   The internet is basically a bunch of virtual cities connected together.  Your local ISP delivers a signal to you via Wireless, Fiber, or some sort of media. When it leaves your house it travels to the ISP’s equipment and is aggregated with your neighbours and sent over faster lines to larger cities. It’s just like a road system. You may get access via a gravel road, which turns into a 2 lane blacktop, which then may turn into a 4 lane highway, and finally a super-highway.  The roads you take depend on where you are going. Your ISP may not have much control over how the traffic flows once it leaves their network.

Bottlenecks can happen anywhere. Anything from fiber optic cuts, oversold capacity, routing issues, and plain old unexpected usage. Why are these important? All of these can affect your speedtest results and can be totally out of control of your ISP and you.  They can also be totally your ISP’s fault. They can also be your fault, just like your car can be.  An underpowered router can be struggling to keep up with your connection. Much like a moped on the above super-highway can’t keep up with a 650 horsepower car to fully utilize the road, your router might not be able to keep up either.  Other things can cause issues such as computer viruses, and low performing components.

Just about any network can become a node or a node with some of the other speedtest sites.  These networks have to meet minimum requirements, but there is no indicator of how utilized these speedtest servers are.  A network could put up one and it’s 100 percent utilized when you go running a speedtest. This doesn’t mean your ISP is slow, just the off-ramp to that speedtest server is slow.

The final thing we want to talk about is the utilization of your internet pipe from your ISP.  This is something most don’t take into consideration.  Let’s go back to our on-ramp analogy.  Your ISP is selling you a connection to the information super-highway.   Say they are selling you a 10 megabyte download connection.  If you have a device in your house streaming an HD Netflix stream, which is typically 5 megs or so, that means you only have 5 megs available for a speedtest while that HD stream is happening. Speedtest only test your current available capacity.  Many folks think a speedtest somehow stops all the traffic on your network, runs the test, and starts the traffic. It doesn’t work that way. A speedtest tests the available capacity at that point in time.  The same is true for any point between you and the speedtest server.  Remember our earlier analogy about slowing down when you got to work because there were so many people trying to get there.  They exceeded the capacity of that destination.  However, that does not mean your connection is necessarily slow because people were zooming past you on their way to less congested destinations.

This is why speedtest results should be taken with a grain of salt. They are a useful tool, but not an absolute. A speedtest server is just a destination.  That destination can have bottlenecks, but others don’t.  Even after this long article, there are many other factors which can affect Internet speed. Things we didn’t touch on like Peering, the technology used, speed limits, and other things can also affect your internet speed to destinations.

Some Random Visio diagram

Below, We have some visio diagrams we have done for customers.

This first design is a customer mesh into a couple of different data centers. We are referring to this as a switch-centric design. This has been talked about in the forums and switch-centric seems like as good as any.

This next design is a netonix switch and a Baicells deployment.

Design for a customer

DMCA Designated Agent Directory updates

The following text is directly from: 

A relevant F.A.Q. can be found at

Service Provider Designation of Agent to Receive Notifications of Claimed Infringement

The Digital Millennium Copyright Act (“DMCA”) provides safe harbors from copyright infringement liability for online service providers. In order to qualify for safe harbor protection, certain kinds of service providers—for example, those that allow users to post or store material on their systems, and search engines, directories, and other information location tools— must designate an agent to receive notifications of claimed copyright infringement. To designate an agent, a service provider must do two things: (1) make certain contact information for the agent available to the public on its website; and (2) provide the same information to the Copyright Office, which maintains a centralized online directory of designated agent contact information for public use. The service provider must also ensure that this information is up to date.

In December 2016, the Office introduced an online registration system and electronically generated directory to replace the Office’s old paper-based system and directory. Accordingly, the Office no longer accepts paper designations. To designate an agent, a service provider must register with and use the Office’s online system.

Transition period: Any service provider that has designated an agent with the Office prior to December 1, 2016, in order to maintain an active designation with the Office, must submit a new designation electronically using the online registration system by December 31, 2017. Any designation not made through the online registration system will expire and become invalid after December 31, 2017. Until then, the Copyright Office will maintain two directories of designated agents: the directory consisting of paper designations made pursuant to the Office’s prior interim regulations which were in effect between November 3, 1998 and November 30, 2016 (the “old directory”), and the directory consisting of designations made electronically through the online registration system (the “new directory”). During the transition period, a compliant designation in either the old directory or the new directory will satisfy the service provider’s obligation under section 512(c)(2) to designate an agent with the Copyright Office. During the transition period, to search for a service provider’s most up-to-date designation, begin by using the new directory. The old directory should only be consulted if a service provider has not yet designated an agent in the new directory.

Antenna Interference issues

Recently, we had a client question why we didn’t mount antennas higher up on a tower with an FM repeater on it. The top of the tower has an FM repeater on it so we mounted the equipment about 25 feet below that.

When you are talking about antennas and transmitters the basic thing to remember is it’s all radiation.   Good antennas have predictable drop off patterns and, when paired with a good transmitter, have crisp frequency drop offs.  However, there is still radiation emitting from feedline and the antenna on the tower. Many FM repeaters use a dipole design.  Some are folded, others are different types.  Below is an antenna pattern from a Dipole antenna.

As you can see there are a few patterns radiating from the antenna.  These patterns should be taken into consideration when mounting your equipment near FM, UHF, or VHF systems. Radiation may interfere with things such as your cat-5, or your PIM.  In an earlier article, I talk about low-pim cables and what affects PIM.  This is very important when you are deploying LTE gear.  RF radiation from high power transmitters can cause PIM issues if the wavelength happens to coincide with the wavelength of the other transmitter.  This does not mean they are on the same frequency.  Remember, in RF you have full wave, 3/4,1/2, and 1/4 wavelengths to deal with.

Other things to consider are near and far field patterns.  If you want some heavy reading you can read about it on Wikipedia.

Our next issue and the most common issue is the radiation getting into our Ethernet cables as well as our radios on the tower.  Below illustrates the propagation of signals coming out of an antenna on the top of the tower.  If you notice, some of the radiation is directed underneath of the antenna. Any equipment mounted too close underneath will be bombarded with radiation.

Too much radiation can cause link negotiation issues,  signal degradation,  and other issues.  By moving our antennas out of the patterns of other antennas we can make for a more reliable system.  This is one case where higher on the tower is not always better. Just because another antenna is not mounted in front of another it does not mean they are in each other’s radiated patterns.

5.8 Cambium Elevated Devices & Max EIRP

Lately, we have had a few clients run into signals becoming worse when they elevated clients to ePMP.   This is not a result of the software being bad, but it enforcing the max EIRP on the units.  This boils down to older devices compliant with original FCC grants which allowed unlimited EIRP.  The Cambium elevate recognizes the latest grant for the devices.  This grant allows for a max of 41 dBM on 5/10/20 mhz channels and 38dBM on 40mhz.

So if you have elevated some older devices from UBNT your signals may have dropped.  This is due to compliance with the latest rules for the device.   As our industry matures, becoming compliant will become more and more important.  On the UBNT units, newer firmware from UBNT also does this.

Cambium has a forum post on this.

We have some tricks of the trade we can do. Contact MTIN for how we can help.

The Importance of cable support in LTE deployments

As the number of WISP LTE deployments increase, there are many things WISPs will need to be mindful of.  One such item is properly supporting antenna cables. LTE systems are more sensitive to cable issues.  In a previous blog post, I talked about pim and low-pim cables.   One of the things that can cause low pim is improperly mated cables.  If cables are not supported they can become loose over time.  Vibration from equipment or even the wind can loosen connections.

How do we support cables?
We can take a cue from the cellular industry. The following are some examples of proper cable support.  Thanks to Joshua Powell for these pics.

Where can you get these?
A good place to start are sites like sitepro1 or Tessco has a selection.

So the next time you are planning your LTE deployment think about cable support.

WISP LTE, PIM testing, and quality

One of the topics that came up during the Baicells troubleshooting tips was the notion of PIM testing, and cables which are PIM rated.

PIM sweeps are a common thing in the Cellular field.   One of the first questions folks often ask is what is a PIM sweep? If you think of PIM testing as a passive test and line sweeping as an active test that is a good start.  PIM testing looks for problems with things like connectors, cables, and other “layer 1” items.  A PIM test is not a line sweep. Line sweeping measures the signal losses and reflections of the transmission system. this is typically VSWR.  A line sweep is an active test. It can not detect the same things a PIM test can.  Many HAM radio folks are familiar with a line sweep where the reflected power is measure in an antenna system. In a line sweep you deal with reflected power and all that.

What does a PIM test do?

When you do a PIM test typical two high power signals are injected into the antenna line.  You can actually pass a sweep test but not a PIM test.

I won’t go into PIM tests very much because you need high dollar units such as those from Anritsu and Kaelus. These cost 10’s of thousands of dollars new.  Sometimes you can find these used.  However, the next thing you will run into is understanding the output of such a device.  Cell crews go to week long certification classes to become a PIM certified tech from Anritsu and others.

What causes a PIM test to fail?

According to Kaelus the most common problems are:

• Contaminated surfaces or contacts due to dirt, dust, moisture or oxidation.
• Loose mechanical junctions due to inadequate torque, poor alignment or poorly prepared contact surfaces.
• Loose mechanical junctions caused transportation shock or vibration .
• Metal flakes or shavings inside RF connections.
• Poorly prepared RF connections
•Trapped dielectric materials (adhesives, foam, etc.)
•Cracks or distortions at the end of the outer conductor of coaxial cables caused by over tightening the back nut during installation.
• Solid inner conductors distorted in the preparation process causing these to be out of round or tapered over the mating length.
• Hollow inner conductors excessively enlarged or made oval during the preparation process.

Why does cable matter?

Cables do not typically cause PIM, but poorly terminated or damaged cables can and do cause problems.

Cables with Seams can cause issues.  The seam can corrode.  Plated copper, found in cheaper cables, can break away from the aluminum core. This actually allows small amounts of flaking to happen between the connector and the core of the cable.  This will cause PIM issues and is very hard to diagnose. Imagine little flakes inside a connector. You don’t see them until you break open the connector, and even then they may be pretty little flakes.

Cables can change their physical configuration as temperature varies. For instance, sunshine can warm cables, changing their electrical length. A cable that happens to be the right length to cancel out PIM when cool may show strong PIM after changing its length on a warm day, or, it can work the other way around, good when hot and bad when cold. In addition, the physical change in length can make a formerly good connection into a poor one, also generating PIM. Other environmental factors such as water in the connector or cable can be an issue, as with any RF setup.

I think I have PIM issues. What are some indications?

PIM often shows up as poor statistics from the affected antenna. One of the first and most direct indications of PIM can be seen in cells with two receive paths. If the noise floor is not equal between the two paths, the cause is likely PIM generated inside the noisy receive path.

How Do I prevent PIM issues?

Cable quality and connector quality are one of the biggest factors in the PIM quality of a LTE system.  Many WISPs are used to making their own LMR cables and putting on their own connectors.  There is a difference between a low PIM LMR-400 cable and normal LMR-400.  Same for connectors.  One of the recommendations today was to use 1/2” superflex heliax.

The easy recommendation is to buy pre-made cables that have already been PIM certified.  In a typical WISP setup, you do not have lots and lot of components in your setup. Buy already certified components from your distributors that are “Low PIM rated”.