Router appears to 'hang' randomly - tips on how to investigate?

Discussions for BiPAC 8900 series: 8900AX-1600, 8900AX-2400, 8900X
nightcustard
Posts: 68
Joined: Sat Nov 03, 2012 2:50 pm

Router appears to 'hang' randomly - tips on how to investigate?

Post by nightcustard »

Hi - I'm running an 8900AX2400 running firmware 2.52.d41 since March 2020 (previous firmware 2.52.d16 since May 2017)

The router has 'hung' three times over the last few weeks. The first time was about six weeks ago, the latter two both occurred this week. The first indication are reports from connected devices of no internet access. On further investigation, I can't ping the router from other LAN devices (hardwired), getting 'General Failure' reported by Windows ping and when I manually assign an IP address and DNS server address (8.8.8.8) to the pinging machine, I get 'Destination host unreachable' reports from ping.

The front panel lights on the router during this 'hang' are as follows: Power - steady green; LAN1 - regularly flashing green (every second or so); DSL - steady green; Internet - green, mostly on but off regularly every two seconds or so. Disconnecting the line input causes the Internet light to remain steady on, whilst the DSL light also remains on. Reconnecting the line input doesn't change the behaviour of the lights (ie) they both remain steady on.

The normal function of the router can be restored by cycling power (waiting 10 secs before reapplying power) but the only log I can see is the start up log. Is there a log showing what happened just before the router stopped working?

Is there anything I can do to distinguish between say, a fault at the ISP end (I think very unlikely given the associated LAN ping failure) or a router hardware fault vs. a router firmware issue? I suspect the latter can only be resolved by reloading the version of firmware I was running previously and waiting but maybe someone can offer a more clever way...

Anyway, here's hoping some nice person can provide some guidance!

Cheers, Mike
billion_fan
Posts: 5383
Joined: Tue Jul 19, 2011 4:30 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by billion_fan »

nightcustard wrote: Thu Jul 30, 2020 2:19 pm Hi - I'm running an 8900AX2400 running firmware 2.52.d41 since March 2020 (previous firmware 2.52.d16 since May 2017)

The router has 'hung' three times over the last few weeks. The first time was about six weeks ago, the latter two both occurred this week. The first indication are reports from connected devices of no internet access. On further investigation, I can't ping the router from other LAN devices (hardwired), getting 'General Failure' reported by Windows ping and when I manually assign an IP address and DNS server address (8.8.8.8) to the pinging machine, I get 'Destination host unreachable' reports from ping.

The front panel lights on the router during this 'hang' are as follows: Power - steady green; LAN1 - regularly flashing green (every second or so); DSL - steady green; Internet - green, mostly on but off regularly every two seconds or so. Disconnecting the line input causes the Internet light to remain steady on, whilst the DSL light also remains on. Reconnecting the line input doesn't change the behaviour of the lights (ie) they both remain steady on.

The normal function of the router can be restored by cycling power (waiting 10 secs before reapplying power) but the only log I can see is the start up log. Is there a log showing what happened just before the router stopped working?

Is there anything I can do to distinguish between say, a fault at the ISP end (I think very unlikely given the associated LAN ping failure) or a router hardware fault vs. a router firmware issue? I suspect the latter can only be resolved by reloading the version of firmware I was running previously and waiting but maybe someone can offer a more clever way...

Anyway, here's hoping some nice person can provide some guidance!

Cheers, Mike
It does seem like a router lock up issue which is most likely a software issue.

Normally we would request a copy of the system log before the issue occurs, hopefully something is logged that will help identify the issue.

As the router locks up there is no way to login to the router to extract the syslog.

The only option is setting up a syslog server, to capture the logs before the lock up occurs.

I have attached a guide on how to do this.

To confirm if it is indeed a software issue, you can always revert to fw 2.52.d16 and see if the same issue occurs again
You do not have the required permissions to view the files attached to this post.
nightcustard
Posts: 68
Joined: Sat Nov 03, 2012 2:50 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by nightcustard »

Thanks billion_fan - I'm assuming the syslog server has to be running when a 'hang' occurs to capture the log entries leading up to the event - is this correct? If so, I'll have a go at setting up a Raspberry Pi syslog server as I don't like leaving my PC on all the time (it burns too much power). I'm not sure how to do this but it looks like a good learning opportunity!

[Edit] I've installed 'syslog-ng' on one of my Pis, following this guide: https://geektechstuff.com/2020/06/13/us ... pberry-pi/ and it appears to be working OK.

Cheers, Mike
Last edited by nightcustard on Mon Aug 03, 2020 11:58 am, edited 1 time in total.
billion_fan
Posts: 5383
Joined: Tue Jul 19, 2011 4:30 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by billion_fan »

nightcustard wrote: Mon Aug 03, 2020 10:35 am Thanks billion_fan - I'm assuming the syslog server has to be running when a 'hang' occurs to capture the log entries leading up to the event - is this correct? If so, I'll have a go at setting up a Raspberry Pi syslog server as I don't like leaving my PC on all the time (it burns too much power). I'm not sure how to do this but it looks like a good learning opportunity!

Cheers, Mike
That's correct the syslog server should capture the logs just before the crash occurs, you will need to install a syslog server on your PI and then set the routers syslog to debugging, (log/display level) and finally enter the IP of the PI as shown the guide I posted.
nightcustard
Posts: 68
Joined: Sat Nov 03, 2012 2:50 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by nightcustard »

Thanks again - now logging (with log rotate even!) and we'll see what pans out....

Cheers, Mike
TapMyHed
Posts: 14
Joined: Fri Nov 01, 2013 2:16 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by TapMyHed »

I've had this same issue. It affects WiFi connected devices only. Running on the d41 firmware.

I've found that if you change a single seeing on the WiFi, 2g pages, you get the message please wait for changes, then both 2g and 5g lights go solid and the WiFi disappears.

You have to power cycle to get it back.

It also randomly happens and generally when I'm watch Amazon prime through my TVs WiFi connection.

I'm just about to restore d23 firmware because that's been the most stable for me.
billion_fan
Posts: 5383
Joined: Tue Jul 19, 2011 4:30 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by billion_fan »

TapMyHed wrote: Wed Aug 05, 2020 8:27 pm I've had this same issue. It affects WiFi connected devices only. Running on the d41 firmware.

I've found that if you change a single seeing on the WiFi, 2g pages, you get the message please wait for changes, then both 2g and 5g lights go solid and the WiFi disappears.

You have to power cycle to get it back.

It also randomly happens and generally when I'm watch Amazon prime through my TVs WiFi connection.

I'm just about to restore d23 firmware because that's been the most stable for me.
Can you submit a ticket on the following link and also provide a backup of your config so I can test your config here

http://www.billion.uk.com/esupport/inde ... ets/Submit
TapMyHed
Posts: 14
Joined: Fri Nov 01, 2013 2:16 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by TapMyHed »

Ok. Will do. On my phone at the moment so will sort it later.
TapMyHed
Posts: 14
Joined: Fri Nov 01, 2013 2:16 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by TapMyHed »

I've rolled back to d27 firmware.

Infinitely better.

I think there are still a few issues with d41 for sure that's making the device annoyingly annoying. One thing I'm glad is gone is the log out function which then brings up the message timed out then stops you logging back in because you always get redirected to the timeout page.

And i also notice that using the device webpage it responds really really quickly. D41 was painfully slow especially after a factory reset and you have to manually put everything back in to ensure it was clean.... Hitting apply was not my favourite thing. I'm down to an hour now though after a factory reset.

I'm going to stick with d27 for now.
billion_fan
Posts: 5383
Joined: Tue Jul 19, 2011 4:30 pm

Re: Router appears to 'hang' randomly - tips on how to investigate?

Post by billion_fan »

TapMyHed wrote: Fri Aug 07, 2020 8:54 am I've rolled back to d27 firmware.

Infinitely better.

I think there are still a few issues with d41 for sure that's making the device annoyingly annoying. One thing I'm glad is gone is the log out function which then brings up the message timed out then stops you logging back in because you always get redirected to the timeout page.

And i also notice that using the device webpage it responds really really quickly. D41 was painfully slow especially after a factory reset and you have to manually put everything back in to ensure it was clean.... Hitting apply was not my favourite thing. I'm down to an hour now though after a factory reset.

I'm going to stick with d27 for now.
Yes best to stick with which firmware works for you
Post Reply