Network troubleshooting conundrum - next steps?
Discussion
Bit of a carry over from the 4G thread, but I have a network situation that is currently doing my head in so thought I would ask on here for suggestions 
My objective is this:
The USG covers routing and DHCP on the network.
The two internet connections are in failover mode on the USG with the ADSL2+ acting as the failover (assuming I can get a stable and fast 4G connection I'll probably ditch this later).
I tested a couple of 1mth sims in the Huawei (stood on its own) from 3 and EE to see what speeds I could get and was happy...so arranged everything per the diagram.
With the 3 sim in the Huawei it all works as expected. I can connect to my network wired or wirelessly from any device and connectivity to the internet is there. I can also connect directly to the Huawei if needed and do the same (not something I intend to do for anything other than troubleshooting).
On EE I'm having trouble though. Which is a massive bummer as the speeds are significantly faster on it.
With the EE sim in the Huawei, the USG obtains an IP address from the Huawei. And the mgmt web page shows the connection on the network is good, as should be the connection between router and network. So all looks good. But there is no internet connectivity from the Unifi network side.
If I connect directly to the Huawei I do get internet connectivity. The devices connected this way get IP addresses the same way as the USG and in the same range etc. So the EE sim is working fine in the router (I think - I do have one glitch on my Windows machine - it will connect to the internet through the wired LAN port but not the Huawei's wifi. I'm assuming at the moment this is something to do with my Windows machine as another laptop I have, a Mac, will connect with both wifi and wired connections direct to the Huawei and get internet connectivity).
As a test, with a degree of faffing about I removed the USG from the mix (removed it from the controller) and connected the Huawei directly to the Unifi network and had it act as the router and DHCP server (controlling the same range as the USG did). This worked for most of the devices on the network, but some it would just not allow to connect to the network at all. Which is obviously not ideal (most devices it was seeing as wired connections...though there were only something like 60 on there max at any one time).
In each of the different configs I've factory reset each of the main devices (USG and Huawei - the ADSL router is actually working stably at the moment. Which is ironic as it was issues with my ADSL link that prompted the 4G exploration!) and configured them fresh after trying them without doing that first.
The ideal objective is to get the network per the diagram with an EE sim (which gives 70-80Mbps download and 10Mbps).
From the messing about I've done thus far I'm basically at the point where nothing is the problem And am failing to see the wood for the trees I think:
The easiest solution would be to ditch EE and just run the 3 sim. But that's defeatist It also doesn't give as marked a speed boost over ADSL as the EE one.
It looks like it might be a routing problem between the 10.0.1.x and 192.168.8.x interfaces on the USG. But why only for the EE sim? Or the IP address of the USG is being blocked from accessing the 4G network by the router? (Firewalls etc are all off on both devices).
Does anyone have any ideas?
My objective is this:
The USG covers routing and DHCP on the network.
The two internet connections are in failover mode on the USG with the ADSL2+ acting as the failover (assuming I can get a stable and fast 4G connection I'll probably ditch this later).
I tested a couple of 1mth sims in the Huawei (stood on its own) from 3 and EE to see what speeds I could get and was happy...so arranged everything per the diagram.
With the 3 sim in the Huawei it all works as expected. I can connect to my network wired or wirelessly from any device and connectivity to the internet is there. I can also connect directly to the Huawei if needed and do the same (not something I intend to do for anything other than troubleshooting).
On EE I'm having trouble though. Which is a massive bummer as the speeds are significantly faster on it.
With the EE sim in the Huawei, the USG obtains an IP address from the Huawei. And the mgmt web page shows the connection on the network is good, as should be the connection between router and network. So all looks good. But there is no internet connectivity from the Unifi network side.
If I connect directly to the Huawei I do get internet connectivity. The devices connected this way get IP addresses the same way as the USG and in the same range etc. So the EE sim is working fine in the router (I think - I do have one glitch on my Windows machine - it will connect to the internet through the wired LAN port but not the Huawei's wifi. I'm assuming at the moment this is something to do with my Windows machine as another laptop I have, a Mac, will connect with both wifi and wired connections direct to the Huawei and get internet connectivity).
As a test, with a degree of faffing about I removed the USG from the mix (removed it from the controller) and connected the Huawei directly to the Unifi network and had it act as the router and DHCP server (controlling the same range as the USG did). This worked for most of the devices on the network, but some it would just not allow to connect to the network at all. Which is obviously not ideal (most devices it was seeing as wired connections...though there were only something like 60 on there max at any one time).
In each of the different configs I've factory reset each of the main devices (USG and Huawei - the ADSL router is actually working stably at the moment. Which is ironic as it was issues with my ADSL link that prompted the 4G exploration!) and configured them fresh after trying them without doing that first.
The ideal objective is to get the network per the diagram with an EE sim (which gives 70-80Mbps download and 10Mbps).
From the messing about I've done thus far I'm basically at the point where nothing is the problem
And am failing to see the wood for the trees I think:- The overall design can be proved to be fine (with a 3 sim in, everything works. Albeit at 25% of the download and 70% of the upload speeds)
- The EE sim will allow internet connectivity in the router, and to multiple devices simultaneously. So in theory the EE sim is fine (I've been on to EE twice today - who were great)
- The USG appears to be doing its job OK. Fails over properly (and quickly)
The easiest solution would be to ditch EE and just run the 3 sim. But that's defeatist
It also doesn't give as marked a speed boost over ADSL as the EE one.It looks like it might be a routing problem between the 10.0.1.x and 192.168.8.x interfaces on the USG. But why only for the EE sim? Or the IP address of the USG is being blocked from accessing the 4G network by the router? (Firewalls etc are all off on both devices).
Does anyone have any ideas?
Edited by Murph7355 on Sunday 24th January 01:19
xeny said:
Have you done traceroutes with the USG present and not present while using the EE SIM? That might at least tell you how far you're getting, and confirm if it is EE's infrastructure not liking the config or something else going on.
No - but was thinking along these lines last night. Just need to figure out how. Will post back when I find out.Thanks
You can try asking in the UI discord too, there’s some experienced people on there
https://discord.gg/Pdp6Dtbr
https://discord.gg/Pdp6Dtbr
Murph7355 said:
The specific one? Or the type? Both use CGNAT I believe.
I'm just curious what IP the "public" interface on the Huawei gets with each SIM.Could there be some kind of IP conflict going on?
Or is one SIM a "voice" SIM but you're trying to use the unlimited data?
I've heard anecdotally of mobile networks doing some f
kery to try and stop that kind of thing.Thanks guys.
Some half interesting tracert stuff:
This is on 3 at the moment. The "user experience" is good in this mode (20-30Mbps download, 5-7 upload at the moment).
C:\WINDOWS\system32> tracert 8.8.8.8
Tracing route to dns.google [8.8.8.8]
over a maximum of 30 hops:
1 2 ms 2 ms 2 ms Gateway [10.0.1.1]
2 2 ms 2 ms 2 ms 192.168.8.1 <<so it's traversing the gateway interfaces quickly>>
3 * * * Request timed out.
4 86 ms 56 ms 56 ms 172.21.37.69 <<not sure what this is as whatsmyip says I'm in the 94.41.208.xxx range as my public IP>>
5 * * * Request timed out.
6 60 ms * 75 ms 172.21.89.98
7 64 ms 58 ms 57 ms 172.21.89.81
8 * * * Request timed out.
9 63 ms 57 ms 58 ms 172.21.106.150
10 56 ms 58 ms 58 ms 185.153.237.155
11 67 ms 57 ms 58 ms 216.239.48.217
12 60 ms 58 ms 57 ms 172.253.71.191
13 65 ms 63 ms 60 ms dns.google [8.8.8.8] <<so that succeeds, albeit taking nearly half a second? NB I'm experiencing no discernible lag that I've noticed>>
I then tried the following which timed out. Which I thought odd as I can access PH OK
C:\WINDOWS\system32> tracert www.pistonheads.com
Tracing route to www.pistonheads.com [13.224.227.32]
over a maximum of 30 hops:
1 2 ms 1 ms 1 ms BoydellsGateway [10.0.1.1]
2 2 ms 1 ms 2 ms 192.168.8.1
3 * * * Request timed out.
4 72 ms 69 ms 62 ms 172.21.37.65
5 * * * Request timed out.
6 * * * Request timed out.
7 57 ms 60 ms 55 ms 172.21.89.81
8 * * * Request timed out.
9 61 ms 79 ms 57 ms 172.21.106.154
10 59 ms 58 ms 57 ms 185.153.238.162
11 62 ms 58 ms 65 ms 185.153.238.163
12 61 ms 63 ms 54 ms 52.94.34.31
13 59 ms 60 ms 53 ms 54.239.101.67
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 56 ms 57 ms 58 ms 150.222.65.99
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
Will play with a few more routes and then switch to EE.
I'm assuming the 172.21.xxx.xxx are likely to be 3 servers of some sort as the ISP?
Some half interesting tracert stuff:
This is on 3 at the moment. The "user experience" is good in this mode (20-30Mbps download, 5-7 upload at the moment).
C:\WINDOWS\system32> tracert 8.8.8.8
Tracing route to dns.google [8.8.8.8]
over a maximum of 30 hops:
1 2 ms 2 ms 2 ms Gateway [10.0.1.1]
2 2 ms 2 ms 2 ms 192.168.8.1 <<so it's traversing the gateway interfaces quickly>>
3 * * * Request timed out.
4 86 ms 56 ms 56 ms 172.21.37.69 <<not sure what this is as whatsmyip says I'm in the 94.41.208.xxx range as my public IP>>
5 * * * Request timed out.
6 60 ms * 75 ms 172.21.89.98
7 64 ms 58 ms 57 ms 172.21.89.81
8 * * * Request timed out.
9 63 ms 57 ms 58 ms 172.21.106.150
10 56 ms 58 ms 58 ms 185.153.237.155
11 67 ms 57 ms 58 ms 216.239.48.217
12 60 ms 58 ms 57 ms 172.253.71.191
13 65 ms 63 ms 60 ms dns.google [8.8.8.8] <<so that succeeds, albeit taking nearly half a second? NB I'm experiencing no discernible lag that I've noticed>>
I then tried the following which timed out. Which I thought odd as I can access PH OK
C:\WINDOWS\system32> tracert www.pistonheads.com
Tracing route to www.pistonheads.com [13.224.227.32]
over a maximum of 30 hops:
1 2 ms 1 ms 1 ms BoydellsGateway [10.0.1.1]
2 2 ms 1 ms 2 ms 192.168.8.1
3 * * * Request timed out.
4 72 ms 69 ms 62 ms 172.21.37.65
5 * * * Request timed out.
6 * * * Request timed out.
7 57 ms 60 ms 55 ms 172.21.89.81
8 * * * Request timed out.
9 61 ms 79 ms 57 ms 172.21.106.154
10 59 ms 58 ms 57 ms 185.153.238.162
11 62 ms 58 ms 65 ms 185.153.238.163
12 61 ms 63 ms 54 ms 52.94.34.31
13 59 ms 60 ms 53 ms 54.239.101.67
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 56 ms 57 ms 58 ms 150.222.65.99
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
Will play with a few more routes and then switch to EE.
I'm assuming the 172.21.xxx.xxx are likely to be 3 servers of some sort as the ISP?
Tracert is always a bit sketchy as to what's really going on and I wouldn't treat it as indicative of anything concrete.
For example I see.
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
1 192.168.0.1 (192.168.0.1) 2.759 ms 1.719 ms 1.880 ms
2 redbus-gw11-10g.idnet.net (212.69.63.36) 14.595 ms 14.617 ms 14.419 ms
3 redbus-gw7-10g.idnet.net (212.69.63.138) 14.903 ms 14.148 ms 14.035 ms
4 redbus-gw6-10g.idnet.net (212.69.63.74) 61.097 ms 58.887 ms 57.530 ms
5 google1.lonap.net (5.57.80.136) 13.798 ms 14.655 ms 14.200 ms
6 74.125.242.65 (74.125.242.65) 14.893 ms 16.017 ms 14.956 ms
7 209.85.248.241 (209.85.248.241) 15.133 ms
172.253.71.191 (172.253.71.191) 15.634 ms
209.85.241.93 (209.85.241.93) 16.130 ms
8 dns.google (8.8.8.8) 15.514 ms 14.476 ms 15.614 ms
traceroute to www.pistonheads.com (13.32.169.47), 64 hops max, 52 byte packets
1 192.168.0.1 (192.168.0.1) 3.191 ms 1.248 ms 1.907 ms
2 redbus-gw11-10g.idnet.net (212.69.63.36) 15.493 ms 14.099 ms 15.228 ms
3 redbus-gw7-10g.idnet.net (212.69.63.138) 14.263 ms 14.566 ms 14.065 ms
4 redbus-gw6-10g.idnet.net (212.69.63.74) 13.539 ms 13.824 ms 14.393 ms
5 lonap-gw-a.amazon.com (5.57.80.8) 20.161 ms 21.006 ms 27.203 ms
6 52.94.32.138 (52.94.32.138) 15.374 ms
52.94.32.144 (52.94.32.144) 15.928 ms
52.94.32.136 (52.94.32.136) 14.937 ms
7 54.239.100.57 (54.239.100.57) 14.692 ms
54.239.100.147 (54.239.100.147) 15.790 ms 15.573 ms
8 52.93.135.29 (52.93.135.29) 16.925 ms
52.93.135.31 (52.93.135.31) 18.412 ms
54.239.42.109 (54.239.42.109) 16.143 ms
What do you see if you try running some ping/tracert commands from on the USG itself via the web interface or console?
Work backwards from the Internet basically
For example I see.
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
1 192.168.0.1 (192.168.0.1) 2.759 ms 1.719 ms 1.880 ms
2 redbus-gw11-10g.idnet.net (212.69.63.36) 14.595 ms 14.617 ms 14.419 ms
3 redbus-gw7-10g.idnet.net (212.69.63.138) 14.903 ms 14.148 ms 14.035 ms
4 redbus-gw6-10g.idnet.net (212.69.63.74) 61.097 ms 58.887 ms 57.530 ms
5 google1.lonap.net (5.57.80.136) 13.798 ms 14.655 ms 14.200 ms
6 74.125.242.65 (74.125.242.65) 14.893 ms 16.017 ms 14.956 ms
7 209.85.248.241 (209.85.248.241) 15.133 ms
172.253.71.191 (172.253.71.191) 15.634 ms
209.85.241.93 (209.85.241.93) 16.130 ms
8 dns.google (8.8.8.8) 15.514 ms 14.476 ms 15.614 ms
traceroute to www.pistonheads.com (13.32.169.47), 64 hops max, 52 byte packets
1 192.168.0.1 (192.168.0.1) 3.191 ms 1.248 ms 1.907 ms
2 redbus-gw11-10g.idnet.net (212.69.63.36) 15.493 ms 14.099 ms 15.228 ms
3 redbus-gw7-10g.idnet.net (212.69.63.138) 14.263 ms 14.566 ms 14.065 ms
4 redbus-gw6-10g.idnet.net (212.69.63.74) 13.539 ms 13.824 ms 14.393 ms
5 lonap-gw-a.amazon.com (5.57.80.8) 20.161 ms 21.006 ms 27.203 ms
6 52.94.32.138 (52.94.32.138) 15.374 ms
52.94.32.144 (52.94.32.144) 15.928 ms
52.94.32.136 (52.94.32.136) 14.937 ms
7 54.239.100.57 (54.239.100.57) 14.692 ms
54.239.100.147 (54.239.100.147) 15.790 ms 15.573 ms
8 52.93.135.29 (52.93.135.29) 16.925 ms
52.93.135.31 (52.93.135.31) 18.412 ms
54.239.42.109 (54.239.42.109) 16.143 ms
What do you see if you try running some ping/tracert commands from on the USG itself via the web interface or console?
Work backwards from the Internet basically
Am only a hop away from the USG. So the 10.0.1.1 is me hitting the USG. Then the 192.168.8.1 is the data hitting the router from the USG. (Will look up how to do the same from the USG, but suspect it will be similar only a smidge quicker).
From a bit of searching the 172... addresses are IANA addresses. And the 185... address in the first trace is 3's addressable range.
So I think this is noting the data gets to the gateway, is passed to the router and then goes out to the internet to start its pathfinding (and in the first trace completes that journey).
Will do the same thing with an EE sim in. If it's them blocking it I guess I'll see something hitting them and then failing every time. If it's the router I suspect it will never escape from 192.168.8.1.
Will also do the same connecting directly to the router.
From a bit of searching the 172... addresses are IANA addresses. And the 185... address in the first trace is 3's addressable range.
So I think this is noting the data gets to the gateway, is passed to the router and then goes out to the internet to start its pathfinding (and in the first trace completes that journey).
Will do the same thing with an EE sim in. If it's them blocking it I guess I'll see something hitting them and then failing every time. If it's the router I suspect it will never escape from 192.168.8.1.
Will also do the same connecting directly to the router.
bhstewie said:
hstewie said: I'd do from the USG it just to rule out any weirdness in how the USG is handling things that sit behind it.
Agree.Am also going to run a few direct from the router.
(OH is having to work a bit today so will wait on those before buggering about with things - much as I'm confident the failover on the USG works
).thebraketester said:
It it work running past the Unifi helpdesk?
Possibly - was also going to pass it by the place I bought the USG from (and have popped a post up on the Huawei site - though they probably already knew I would ).Will run a few traceroutes to see if it shows anything up and then broaden the support net. (My OH keeps telling me just to leave the 3 sim in
).Put the EE sim in.
First go gave me traceroutes through the failover connection (JL Broadband)...the USG doing its job there.
Disconnecting the JL Broadband connection just resulted in the traceroutes timing out. Didn't even give the 4G routers address as a hop.
Tried it from the USG directly with the same results.
Connected my OH's laptop up directly to the 4G router and that gave the following:
traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
1 homerouter.cpe (192.168.8.1) 2.565 ms !N 1.426 ms !N *
(It's a Mac, so not sure if it can give anything more verbose, but that's all the trace gave. Internet connectivity works fine though.)
So tried my laptop (Win10). The tracert command on that is timing out despite internet connectivity. Interestingly searching whats my ip on Google returns 2a01:4c8:1061:1feb:98e:469c:27a:53ad (!). It also does on my OH's Mac.
Is this an IPv6 IP address? The 3 sim gave an IPv4 IP address, so am wondering if this is something to do with the problem.
The router seems to be OK handling it as when I connect directly I get internet access. But something seems to be going amiss when the USG is connected - which is a bit odd as I would have thought it would talk to the router via the IPv4 address it's given, letting the router go from its IPv4 range to IPv6?
More Googling required.
First go gave me traceroutes through the failover connection (JL Broadband)...the USG doing its job there.
Disconnecting the JL Broadband connection just resulted in the traceroutes timing out. Didn't even give the 4G routers address as a hop.
Tried it from the USG directly with the same results.
Connected my OH's laptop up directly to the 4G router and that gave the following:
traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
1 homerouter.cpe (192.168.8.1) 2.565 ms !N 1.426 ms !N *
(It's a Mac, so not sure if it can give anything more verbose, but that's all the trace gave. Internet connectivity works fine though.)
So tried my laptop (Win10). The tracert command on that is timing out despite internet connectivity. Interestingly searching whats my ip on Google returns 2a01:4c8:1061:1feb:98e:469c:27a:53ad (!). It also does on my OH's Mac.
Is this an IPv6 IP address? The 3 sim gave an IPv4 IP address, so am wondering if this is something to do with the problem.
The router seems to be OK handling it as when I connect directly I get internet access. But something seems to be going amiss when the USG is connected - which is a bit odd as I would have thought it would talk to the router via the IPv4 address it's given, letting the router go from its IPv4 range to IPv6?
More Googling required.
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff