R7000 : Hangs every 12 hours

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2  Next
Author Message
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Sat Apr 17, 2021 6:48    Post subject: R7000 : Hangs every 12 hours Reply with quote
My r7000 (with firmware : DD-WRT v3.0-r45955 std (03/09/21)) appears to hang every 12 hours. This is AFTER I introduced Pfsense into my network. A reboot fixes the issue temporarily.

Ive tried defaulting the firmware to its factory setting but the issue still persists.

Note : If I connect my ISP`s cable directly to the R7000, I dont see any issues.

My Current Setup at home
Internet ---> Pfsense ---> DDwrt ---> Clients

I followed some instructions on how to disable WAN, assign it to the switch, disable the firewall, disable DHCP, etc

Post hanging you would see symptoms like:
1) Pings stop working
2) Wifi and LAN dont work
3) GUI is in-accessible

I read somewhere that it might be possible that my R7000 is overwhelmed by the traffic coming from pfsense, and I`m not sure how to remedy that. I did also happen to read an article about increasing the Maximum ports, and decreasing the timeouts for TCP.

These are my current settings:

TCP Congestion Control : westwood

Maximum Ports : 32768
(Default: 4096, Range: 256 - 65536)

TCP Timeout (in seconds): 3600
(Default: 3600, Range: 1 - 86400)

UDP Timeout (in seconds) : 120

I enabled sysLog to see if I could catch anything (Dropped or Refused connections). But around the times it hangs, I dont see anything useful that would point to a cause.
Just a bunch of these that could mean anything:

Apr 17 11:20:19 daemon.err ntpclient[3483]: Failed resolving address to hostname 2.pool.ntp.org: Try again
Apr 17 11:20:19 daemon.err ntpclient[3483]: Failed resolving server 2.pool.ntp.org: Network is down

Can anyone please guide me as to how to diagnose the problem and fix it?
Sponsor
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12836
Location: Netherlands

PostPosted: Sat Apr 17, 2021 7:51    Post subject: Reply with quote
If you setup as a WAP the router is acting as a dumb switch so if setup correctly it is unlikely it is overpowered.

My advice: reset to defaults and use the proper instructions to setup as a WAP: https://wiki.dd-wrt.com/wiki/index.php/Wireless_access_point

To be clear after resetting do not restore from a backup file, put settings in manually

_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Sat Apr 17, 2021 8:37    Post subject: Reply with quote
Many thanks egc.

If you setup as a WAP the router is acting as a dumb switch so if setup correctly it is unlikely it is overpowered.
--> I thought so too, which is why I didnt mess around with any of the default settings

My advice: reset to defaults and use the proper instructions to setup as a WAP: https://wiki.dd-wrt.com/wiki/index.php/Wireless_access_point
--> Hah, I can already see some steps I missed....Namely:

1) Open the Setup -> Advanced Routing tab
(Recommended) Change operating mode to: Router, then Save

2) Open the Security -> Firewall tab
Uncheck all boxes except Filter Multicast, then Save

The rest of it is in place. Will test these and get back.

To be clear after resetting do not restore from a backup file, put settings in manually
--> I didnt. Clean reset, with no restore.
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Sun Apr 18, 2021 6:33    Post subject: Reply with quote
egc wrote:
If you setup as a WAP the router is acting as a dumb switch so if setup correctly it is unlikely it is overpowered.

My advice: reset to defaults and use the proper instructions to setup as a WAP: https://wiki.dd-wrt.com/wiki/index.php/Wireless_access_point

To be clear after resetting do not restore from a backup file, put settings in manually


Hi Egc, The router crashed again last night, close to midnight. Same symptoms as before.

Any leads on logs to go through? Curious to see whats causing it. The Syslogs strangely dont seem to have anything useful. And the dates (though NTP is setup correctly) seem incorrect or missing.
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Mon Apr 19, 2021 14:46    Post subject: Reply with quote
After rebooting the router, I`ve observed something strange : The router now only crashes every 24 hours....exactly at midnight. Step in the right direction I feel.

I`d still like to know if there are any logs I could setup on a remote server for this. After every reboot I notice the logs for the previous day vanish. Would be interesting to see what makes it hang.
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12836
Location: Netherlands

PostPosted: Mon Apr 19, 2021 15:08    Post subject: Reply with quote
You can see: https://wiki.dd-wrt.com/wiki/index.php/Logging_with_DD-WRT
_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 14125
Location: Texas, USA

PostPosted: Mon Apr 19, 2021 16:39    Post subject: Reply with quote
Or you can tail the syslog in PuTTY as well as the serial console. You may be able to glean more information by setting console debug active as well:
BrainSlayer wrote:
go into cfe console and do the following 

nvram set console_debug=1 
nvram commit 
reboot 

_________________
"Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT
Pogo - A minimal level of ability is expected and needed...
DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)

----------------------
Linux User #377467 counter.li.org / linuxcounter.net
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Mon Apr 19, 2021 17:20    Post subject: Reply with quote
Thanks guys. Ive kept Putty to tail ddwrt and.....syslog watcher too. The hang should happen about an hour from now. Ill report back, once I`m done.
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Tue Apr 20, 2021 5:16    Post subject: Reply with quote
kernel-panic69 wrote:
Or you can tail the syslog in PuTTY as well as the serial console. You may be able to glean more information by setting console debug active as well:
BrainSlayer wrote:
go into cfe console and do the following 

nvram set console_debug=1 
nvram commit 
reboot 


As usual the router crashed exactly at 12 Midnight. The remote server happened to capture this, but i`m not sure what this means:

20-04-2021 00:17 Info (CRON) STARTUP (fork ok)
20-04-2021 00:17 Info cron : daemon successfully started
20-04-2021 00:17 Notice klogd started: BusyBox v1.33.0 (2021-03-09 12:03:56 +07)
20-04-2021 00:17 Info wland : daemon successfully started
20-04-2021 00:17 Info syslogd : syslog daemon successfully stopped
20-04-2021 00:17 Info telnetd : daemon successfully started
20-04-2021 00:17 Notice klogd: exiting
20-04-2021 00:17 Info klogd : kernel log daemon successfully stopped
20-04-2021 00:17 Info device br0 left promiscuous mode
20-04-2021 00:00 Alert fast-classifier: registered
20-04-2021 00:00 Alert fast-classifier (PBR safe v2.1.6b): starting up
20-04-2021 00:00 Info sfe : shortcut forwarding engine successfully started
20-04-2021 00:00 Info sfe : shortcut forwarding engine successfully stopped
20-04-2021 00:00 Alert fast-classifier: shutting down

Prior to 00:00 theres just regular logs, which dont help much.

I read a similiar article here : https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=960882

Fella had some luck with an NVram erase. But before I try that, I think i`ll try the debug method above, and wait for it to crash again.
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Wed Apr 21, 2021 11:26    Post subject: Reply with quote
Crashed again at 12 Midnight

Here are some of the logs from syslog watcher: (Older entries start from the bottom)

21-04-2021 00:00 Info _evalpid:ifconfig br0 down
21-04-2021 00:00 Info redial : ppp redial daemon trying to stop
21-04-2021 00:00 Info ntpclient : Network Time Protocol client trying to stop
21-04-2021 00:00 Info sputnik : daemon trying to stop
21-04-2021 00:00 Info listen : activity daemon trying to stop
21-04-2021 00:00 Info pptp : daemon trying to stop
21-04-2021 00:00 Info pppd : daemon trying to stop
21-04-2021 00:00 Debug route_del: if: dst: gw: (null) mask: (null)
21-04-2021 00:00 Info bpalogin : bpalogin trying to stop
21-04-2021 00:00 Info listen : connectivity listener trying to stop
21-04-2021 00:00 Info xl2tpd : daemon trying to stop
21-04-2021 00:00 Info pppd : daemon trying to stop
21-04-2021 00:00 Debug route_del: if: dst: gw: (null) mask: (null)
21-04-2021 00:00 Info pppd : pppoe daemon trying to stop
21-04-2021 00:00 Info pppd : pppoe process trying to stop
21-04-2021 00:00 Info _evalpid:wrtbwmon setup /tmp/bw.db
21-04-2021 00:00 Info _evalpid:rm -f /tmp/.fw_lock
21-04-2021 00:00 Info sfe : shortcut forwarding engine successfully started
21-04-2021 00:00 Alert fast-classifier: registered
21-04-2021 00:00 Alert fast-classifier (PBR safe v2.1.6b): starting up
21-04-2021 00:00 Debug system2:echo 1 > /sys/skip_to_bridge_ingress
21-04-2021 00:00 Info _evalpid:insmod fast-classifier
21-04-2021 00:00 Info _evalpid:insmod shortcut-fe-ipv6
21-04-2021 00:00 Info _evalpid:insmod shortcut-fe
21-04-2021 00:00 Info _evalpid:rmmod xt_ipp2p
21-04-2021 00:00 Info _evalpid:rmmod ipt_ipp2p
21-04-2021 00:00 Info _evalpid:rmmod xt_layer7
21-04-2021 00:00 Info _evalpid:rmmod ipt_layer7
21-04-2021 00:00 Info _evalpid:rmmod ipt_webstr
21-04-2021 00:00 Info _evalpid:iptables -t mangle -F
21-04-2021 00:00 Info _evalpid:iptables -t nat -F
21-04-2021 00:00 Info _evalpid:iptables -F
21-04-2021 00:00 Info sfe : shortcut forwarding engine successfully stopped
21-04-2021 00:00 Info _evalpid:rmmod shortcut-fe
21-04-2021 00:00 Info _evalpid:rmmod shortcut-fe-ipv6
21-04-2021 00:00 Alert fast-classifier: shutting down
21-04-2021 00:00 Info _evalpid:rmmod fast-classifier
21-04-2021 00:00 Info _evalpid:iptables -t raw -F

Lookks like a bunch of services going into a stop state. Dont see anything in the prior logs. Ill paste those below anyway, if anyone can see something I cannot:

20-04-2021 23:59 Info dhcpfwd : dhcp forwarder trying to stop
20-04-2021 23:59 Info _evalpid:/sbin/service wan stop
20-04-2021 23:53 Info _evalpid:filtersync
20-04-2021 23:53 Info cyclic NTP Update success (servers 2.pool.ntp.org 212.18.3.19(ntp2.m-online.net) 88.99.174.22(static.88-99-174-22.clients.your-server.de))
20-04-2021 23:53 Info Time set from 212.18.3.19(ntp2.m-online.net) [212.18.3.19(ntp2.m-online.net)].
20-04-2021 23:53 Debug Connecting to 212.18.3.19(ntp2.m-online.net) [212.18.3.19(ntp2.m-online.net)] ...
20-04-2021 23:53 Notice Network up resolved address to hostname 212.18.3.19(ntp2.m-online.net)
20-04-2021 23:53 Error Failed resolving server 2.pool.ntp.org: Network is down
20-04-2021 23:53 Error Failed resolving address to hostname 2.pool.ntp.org: Try again
20-04-2021 23:53 Info _evalpid:ntpclient 2.pool.ntp.org 212.18.3.19(ntp2.m-online.net) 88.99.174.22(static.88-99-174-22.clients.your-server.de)
20-04-2021 23:53 Info ntpclient : Network Time Protocol client trying to stop
20-04-2021 23:53 Info _evalpid:/sbin/service ntpc stop
20-04-2021 23:53 Info _evalpid:stopservice ntpc -f
20-04-2021 22:53 Info _evalpid:filtersync
20-04-2021 22:53 Info cyclic NTP Update success (servers 2.pool.ntp.org 212.18.3.19(ntp2.m-online.net) 88.99.174.22(static.88-99-174-22.clients.your-server.de))
20-04-2021 22:53 Info Time set from 212.18.3.19(ntp2.m-online.net) [212.18.3.19(ntp2.m-online.net)].
20-04-2021 22:53 Debug Connecting to 212.18.3.19(ntp2.m-online.net) [212.18.3.19(ntp2.m-online.net)] ...
20-04-2021 22:53 Notice Network up resolved address to hostname 212.18.3.19(ntp2.m-online.net)
20-04-2021 22:53 Error Failed resolving server 2.pool.ntp.org: Network is down
20-04-2021 22:53 Error Failed resolving address to hostname 2.pool.ntp.org: Try again
20-04-2021 22:53 Info _evalpid:ntpclient 2.pool.ntp.org 212.18.3.19(ntp2.m-online.net) 88.99.174.22(static.88-99-174-22.clients.your-server.de)
20-04-2021 22:53 Info ntpclient : Network Time Protocol client trying to stop
20-04-2021 22:53 Info _evalpid:/sbin/service ntpc stop
20-04-2021 22:53 Info _evalpid:stopservice ntpc -f
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12836
Location: Netherlands

PostPosted: Wed Apr 21, 2021 12:15    Post subject: Reply with quote
It could be that your network goes down triggering a router reboot.

You do not have the connection watch dog enabled?
(Administration/Keepalive)

_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Wed Apr 21, 2021 12:43    Post subject: Reply with quote
egc wrote:
It could be that your network goes down triggering a router reboot.

You do not have the connection watch dog enabled?
(Administration/Keepalive)


The way its setup here is :

Internet (ISP) ---> Pfsense (PPPoe Auth + Security + DHCP)---> DDwrt (LAN + WIFI AP) ---> Clients

I checked the gateway logs on the pfsense box, at 12 Midnight. No sign of the Network going down. Its just strange that now it stops working on the dot of 12AM.

Its not a reboot really. The ddwrt router just hangs.

I tried the KeepALive option earlier, not really reliable.
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 12836
Location: Netherlands

PostPosted: Wed Apr 21, 2021 13:45    Post subject: Reply with quote
I would just do from CLI :
Code:
nvram erase && reboot


and setup manually, starting with only minimal settings according to the wiki

_________________
Routers:Netgear R7000, R6400v1, R6400v2, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 6408
Location: UK, London, just across the river..

PostPosted: Wed Apr 21, 2021 13:50    Post subject: Reply with quote
Do you by any chance, have a name for NTP server in the NTP time box..? Use IP instead of name or just try some other DDWRT NTP zone selection...if not use like Cloudflare NTP IP 162.159.200.123 or google NTP IP 216.239.35.8
I had something funny like this in the past, the NTP i used was sending a reboot signal at 0:00h or god knows what and how, but my router was doing the same crap...changed to goggle NTP time IP and happy days...

The other thing that comes to my mind is, if you have a cron script or other scripts that mess up with time...somehow...

egc wrote:
I would just do from CLI :
Code:
nvram erase && reboot


and setup manually, starting with only minimal settings according to the wiki


I do support that option too.. Laughing

_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 55179 WAP
TP-Link WR1043NDv2 -DD-WRT 55303 Gateway/DoT,Forced DNS,Ad-Block,Firewall,x4VLAN,VPN
TP-Link WR1043NDv2 -Gargoyle OS 1.15.x AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear XR500 --DD-WRT 55460 Gateway/DoH,Forced DNS,AP Isolation,4VLAN,Ad-Block,Firewall,Vanilla
Netgear R7800 --DD-WRT 55460 Gateway/DoT,AD-Block,Forced DNS,AP&Net Isolation,x3VLAN,Firewall,Vanilla
Netgear R9000 --DD-WRT 55363 Gateway/DoT,AD-Block,AP Isolation,Firewall,Forced DNS,x2VLAN,Vanilla
Broadcom
Netgear R7000 --DD-WRT 55460 Gateway/SmartDNS/DoH,AD-Block,Firewall,Forced DNS,x3VLAN,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby DNS over TLS I DNSCrypt v2 by mac913
mythz80
DD-WRT Novice


Joined: 08 Apr 2016
Posts: 19

PostPosted: Wed Apr 21, 2021 14:43    Post subject: Reply with quote
Thanks EGC and Alozaros.

The NTP settings > Server IP/Name field is blank.

@Alozaros : Thats very interesting ..... Did you figure it out through a set of logs?

I `m going to fill the "Server IP/Name" field with a Google NTP IP and see what happens. Followed by disabling it completely (just in case the prior step does nothing). Eh, besides, the ddwrt routers only an AP atm, so I dont see any harm. Based on what you experienced.....and those constant NTP errors you guys saw in the logs, this might be worth trying.

If all else fails, Ill do the NVram erase bit. Wink
Goto page 1, 2  Next Display posts from previous:    Page 1 of 2
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum