Network Data Loops
Author
Discussion

PugwasHDJ80

Original Poster:

7,684 posts

252 months

Wednesday 9th January 2019
quotequote all
Just received this from our IT department:

"Over the past 12 months the business has experienced several widescale outages (total loss of internet connectivity).

These have been caused by users plugging network cables into wall mounted network ports and creating a data loop (data storm) which crashes the system.

This type of incident is very difficult to diagnose, and once the ‘data storm’ effect has been identified requires ALL DEVICES on the network to be unplugged and then plugged in individually to locate the problem device."

So, whats really going on? if you plug in to an ethernet port, i'd expect it to be either unpatched, or if it is patched, that it tries to negotiate an IP address (and fails because it doesnt' have a registered MAC address on the network). I can't see how it can create a data loop of any sort if you're using a basic STP or SPB protocol? what have i missed?

LordGrover

34,189 posts

243 months

Wednesday 9th January 2019
quotequote all
I don't know why or the technicalities, but I know it happens. We had similar when someone linked two ports on a switch.

ooo000ooo

2,648 posts

225 months

Wednesday 9th January 2019
quotequote all
Not an expert - looping 2 ports should only knock off those 2 port. On our network which is bodged together by amateurs, duct tape and loo rolls. we got problems when someone plugged in another switch or similar that hadn't been configured properly. IIRC spanning tree hadn't been enabled?

Garemberg

469 posts

120 months

Wednesday 9th January 2019
quotequote all
Those ports are configured as edge ports, so won't participate in RSTP/STP. user plugs in a cable - creates a loop.

it's pretty basic stuff in this day and age tbh. also all unused switchports should be shutdown.


any idea what the switches are

captain_cynic

17,648 posts

126 months

Wednesday 9th January 2019
quotequote all
PugwasHDJ80 said:
Just received this from our IT department:

"Over the past 12 months the business has experienced several widescale outages (total loss of internet connectivity).

These have been caused by users plugging network cables into wall mounted network ports and creating a data loop (data storm) which crashes the system.

This type of incident is very difficult to diagnose, and once the ‘data storm’ effect has been identified requires ALL DEVICES on the network to be unplugged and then plugged in individually to locate the problem device."

So, whats really going on? if you plug in to an ethernet port, i'd expect it to be either unpatched, or if it is patched, that it tries to negotiate an IP address (and fails because it doesnt' have a registered MAC address on the network). I can't see how it can create a data loop of any sort if you're using a basic STP or SPB protocol? what have i missed?
They aren't lying to you, there's no conspiracy here and this is why you never plug anything in without being told to (also why an IT dept should never leave cables lying about, especially near empty ports).

What happens when you plug a switch into a switch is that both ports immediately start sending out ARP (Address Resolution Protocol) updates. ARP is a layer below IP addresses.

When a switch gets new ARP information it sends out that information to it's peers. When you've got two ports sending out the same information it forms a switching loop which continually sends out ARP updates as they try to correct each other, in turn the other switch ports re-transmit this information flooding the network with packets.

juice

9,935 posts

313 months

Wednesday 9th January 2019
quotequote all
Implement SNMP traps on the switch.
Audit your ports
Turn off unused ones.


kev1974

4,030 posts

160 months

Wednesday 9th January 2019
quotequote all
No way can you "create a data loop" that takes down the entire network, just by plugging in a cable or computer on a cable.

I guess you could if you plugged BOTH ends of the cable into wall sockets, if they were both patched in, and the network equipment is from the 1990s.

Sounds like an IT department that is either clueless, or thinks spreading fear is the way to enforce policy.

WinstonWolf

72,863 posts

270 months

Wednesday 9th January 2019
quotequote all
kev1974 said:
No way can you "create a data loop" that takes down the entire network, just by plugging in a cable or computer on a cable.

I guess you could if you plugged BOTH ends of the cable into wall sockets, if they were both patched in, and the network equipment is from the 1990s.

Sounds like an IT department that is either clueless, or thinks spreading fear is the way to enforce policy.
Well other than the fact that depending on the equipment you absolutely can, that's a pretty accurate post thumbup

It all depends on what's in the wiring closet...

HantsRat

2,411 posts

139 months

Wednesday 9th January 2019
quotequote all
Implement 802.1X on the network with a fallback to an unauthenticated VLAN.

AllyBassman

779 posts

143 months

Wednesday 9th January 2019
quotequote all
It can happen, I remember having to hunt down the root cause of a loop numerous times at the start of my career while working as IT tech in a large high school. If caught early it wouldn't be an issue, if left too long it would swamp your LAN and indeed take down your network.

Most decent Layer 3 / 4 switches have loop protection these days, so i can only assume they're using ancient kit!

WinstonWolf

72,863 posts

270 months

Wednesday 9th January 2019
quotequote all
AllyBassman said:
It can happen, I remember having to hunt down the root cause of a loop numerous times at the start of my career while working as IT tech in a large high school. If caught early it wouldn't be an issue, if left too long it would swamp your LAN and indeed take down your network.

Most decent Layer 3 / 4 switches have loop protection these days, so i can only assume they're using ancient kit!
Another favourite was customers unplugging BNC connections to move their desk around, not realising that they'd just bought the entire network to halt rofl

LordGrover

34,189 posts

243 months

Wednesday 9th January 2019
quotequote all
Thems was the dayses was thems.

WinstonWolf

72,863 posts

270 months

Wednesday 9th January 2019
quotequote all
LordGrover said:
Thems was the dayses was thems.
Make before break connectors were the future I tell ya' biggrin