{"id":2662,"date":"2018-11-30T05:58:54","date_gmt":"2018-11-30T05:58:54","guid":{"rendered":"http:\/\/www.mtin.net\/blog\/?p=2662"},"modified":"2018-11-30T05:58:54","modified_gmt":"2018-11-30T05:58:54","slug":"the-importance-of-network-monitoring-systems-nms","status":"publish","type":"post","link":"https:\/\/www.mtin.net\/blog\/the-importance-of-network-monitoring-systems-nms\/","title":{"rendered":"The importance of Network Monitoring Systems (NMS)"},"content":{"rendered":"<p>One of our open tickets on <a href=\"https:\/\/www.midwest-ix.com\">MidWest-IX<\/a> is a member reporting slow speeds on their exchange port. After having them send us some data and a few e-mails back and forth we began looking at their switch port on the fabric.\u00a0 Right away we noticed errors on the port. After a counter reset the errors were still incrementing<\/p>\n<pre>\u00a019 runts\u00a0 0 giants\u00a0 1210 CRC\u00a0 0 no buffer\r\n\u00a01329 input error\u00a0 0 short frame\u00a0 0 overrun\u00a0 0 underrun\u00a0 0 ignored<\/pre>\n<p>This led us to look at our <a href=\"https:\/\/www.librenms.org\/\">LibreNMS<\/a> data for this port.\u00a0 A quick look shows on October 31st the port started seeing input errors.<\/p>\n<p><img data-attachment-id=\"2663\" data-permalink=\"https:\/\/www.mtin.net\/blog\/the-importance-of-network-monitoring-systems-nms\/graph-1-4\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?fit=1521%2C467&amp;ssl=1\" data-orig-size=\"1521,467\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"graph-1\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?fit=300%2C92&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?fit=580%2C178&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-2663 size-large\" src=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?resize=580%2C178\" alt=\"\" width=\"580\" height=\"178\" srcset=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?resize=1024%2C314&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?resize=300%2C92&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?w=1521&amp;ssl=1 1521w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?w=1160 1160w\" sizes=\"(max-width: 580px) 100vw, 580px\" data-recalc-dims=\"1\" \/><\/p>\n<p>By dilling down we are able to see exactly when this started happening<\/p>\n<p><img data-attachment-id=\"2664\" data-permalink=\"https:\/\/www.mtin.net\/blog\/the-importance-of-network-monitoring-systems-nms\/graph-2-4\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?fit=1521%2C467&amp;ssl=1\" data-orig-size=\"1521,467\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"graph-2\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?fit=300%2C92&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?fit=580%2C178&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-2664\" src=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?resize=580%2C178\" alt=\"\" width=\"580\" height=\"178\" srcset=\"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?resize=1024%2C314&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?resize=300%2C92&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?w=1521&amp;ssl=1 1521w, https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-2.png?w=1160 1160w\" sizes=\"(max-width: 580px) 100vw, 580px\" data-recalc-dims=\"1\" \/><\/p>\n<p>We now have responded to the customer to see if anything changed that day. Maybe a new switch, new optic, or software upgrade.\u00a0 By having this data available in an NMS we were able to cut down on troubleshooting by a huge margin.\u00a0 We now know when the issue started and are closer to the root cause of this.\u00a0 Without this data, we would be spending more time trying to diagnose and track down issues.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One of our open tickets on MidWest-IX is a member reporting slow speeds on their exchange port. After having them send us some data and a few e-mails back and forth we began looking at their switch port on the fabric.\u00a0 Right away we noticed errors on the port. After a counter reset the errors [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2663,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[17],"tags":[671,670,672,92],"jetpack_publicize_connections":[],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2018\/11\/graph-1.png?fit=1521%2C467&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6VLMf-GW","jetpack-related-posts":[{"id":2277,"url":"https:\/\/www.mtin.net\/blog\/winbox-brute-force\/","url_meta":{"origin":2662,"position":0},"title":"Winbox brute Force","author":"j2sw","date":"June 1, 2018","format":false,"excerpt":"You really should not have your winbox port open to anything but a management network, but if you need a script to help with brute force on the Mikrotik. add action=drop chain=input comment=\"drop winbox brute forcers\" dst-port=8291 \\ protocol=tcp src-address-list=winbox_blacklist add action=add-src-to-address-list address-list=winbox_blacklist \\ address-list-timeout=1w3d chain=input connection-state=new dst-port=8291 \\ protocol=tcp\u2026","rel":"","context":"Similar post","block_context":{"text":"Similar post","link":""},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2755,"url":"https:\/\/www.mtin.net\/blog\/basic-ipv6-mikrotik-firewall\/","url_meta":{"origin":2662,"position":1},"title":"Basic IPV6 Mikrotik Firewall","author":"j2sw","date":"January 24, 2019","format":false,"excerpt":"Below is a basic IPV6 firewall fillter for your Mikrotik CPE devices.\u00a0 This is a good start for customer-facing CPE. \u00a0 \/ipv6 firewall filter add chain=forward comment=\"allow forwarding established, related\" connection state=established,related add chain=forward comment=\"allow forward lan->wan\" in-interface=lan out-interface=wan add chain=forward comment=\"allow ICMPv6 forwarding\" in-interface=wan protocol=icmpv6 add action=reject chain=forward comment=\"reject\u2026","rel":"","context":"In &quot;IPV6&quot;","block_context":{"text":"IPV6","link":"https:\/\/www.mtin.net\/blog\/category\/networking\/ipv6\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1537,"url":"https:\/\/www.mtin.net\/blog\/the-importance-of-checking-layer-one-and-two\/","url_meta":{"origin":2662,"position":2},"title":"The importance of checking layer one and two","author":"j2sw","date":"April 18, 2017","format":false,"excerpt":"I had a simple network consisting of a Mikrotik hooked to an internet connection along with 3 APs behind it. \u00a0Nothing fancy, \u00a0The network was experiencing drop out in service. \u00a0The internet would just stop. \u00a0One of the most noticeable things would iPhones would drop the wireless link and revert\u2026","rel":"","context":"In &quot;Mikrotik&quot;","block_context":{"text":"Mikrotik","link":"https:\/\/www.mtin.net\/blog\/category\/mikrotik\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.mtin.net\/blog\/wp-content\/uploads\/2017\/04\/Screen-Shot-2017-04-18-at-12.18.35-AM-2-300x197.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":297,"url":"https:\/\/www.mtin.net\/blog\/protecting-your-mikrotik-from-dns-amplification\/","url_meta":{"origin":2662,"position":3},"title":"Protecting your Mikrotik from DNS Amplification","author":"j2sw","date":"May 8, 2015","format":false,"excerpt":"There are several reasons and benefits to using your Mikrotik as a DNS caching server. \u00a0Queries to the client are just a tad faster, which makes the overall user experience seem snappier. \u00a0It also allows you to quickly change upstream DNS servers in the even of an outage, attack, etc.\u2026","rel":"","context":"In \"amplification\"","block_context":{"text":"amplification","link":"https:\/\/www.mtin.net\/blog\/tag\/amplification\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2165,"url":"https:\/\/www.mtin.net\/blog\/ipv6-firewall-rules-for-mikrotik\/","url_meta":{"origin":2662,"position":4},"title":"IPV6 Firewall rules for Mikrotik","author":"j2sw","date":"March 23, 2018","format":false,"excerpt":"Some basic IPV6 Firewall Rules for Mikrotik. Replace in-interface=\"\" with your appropriate interface. \/ipv6 firewall filter add chain=input protocol=icmpv6 add chain=input connection-state=established,related add chain=input dst-port=546 in-interface=ether1-wan protocol=udp src-port=547 add action=drop chain=input connection-state=invalid add action=drop chain=input connection-state=new in-interface=ether1-wan add chain=forward protocol=icmpv6 add chain=forward connection-state=established,related add chain=forward connection-state=new in-interface=!ether1-wan add action=drop chain=forward\u2026","rel":"","context":"In &quot;Mikrotik&quot;","block_context":{"text":"Mikrotik","link":"https:\/\/www.mtin.net\/blog\/category\/mikrotik\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":206,"url":"https:\/\/www.mtin.net\/blog\/equinix-exchange-port-faq\/","url_meta":{"origin":2662,"position":5},"title":"Equinix Exchange Port FAQ","author":"j2sw","date":"November 18, 2014","format":false,"excerpt":"For customers ordering ports on the Equinix exchange https:\/\/ix.equinix.com\/ixp\/FAQ","rel":"","context":"In &quot;Data Center&quot;","block_context":{"text":"Data Center","link":"https:\/\/www.mtin.net\/blog\/category\/data-center\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/posts\/2662"}],"collection":[{"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/comments?post=2662"}],"version-history":[{"count":1,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/posts\/2662\/revisions"}],"predecessor-version":[{"id":2665,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/posts\/2662\/revisions\/2665"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/media\/2663"}],"wp:attachment":[{"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/media?parent=2662"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/categories?post=2662"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mtin.net\/blog\/wp-json\/wp\/v2\/tags?post=2662"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}