RT-AC3100: top 30% - 50%+ sirq : Loss of connection.

Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware
Goto page 1, 2  Next
Author Message
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sun Nov 13, 2022 21:32    Post subject: RT-AC3100: top 30% - 50%+ sirq : Loss of connection. Reply with quote
Hey Folks,

I'm connected as follows Laptop -> Wireless to -> Asus Router #1 -> Wired to -> Asus Router Internet -> WWW.

When browsing various pages on my laptop, the 'Asus Internet (192.168.0.6)' routers sirq spikes, then at times the router is inaccessible or shows a ping of 3+ seconds. The sirq shoots up to:

30% - 50%+ sirq

All my routers are on firmware r49392. Wondering, what could be the issue here and how to find more information on this?

Cheers,
TK


Code:

Pre-authentication banner message from server:
| DD-WRT v3.0-r49392 std (c) 2022 NewMedia-NET GmbH
| Release: 06/29/22
| Board: Asus RT-AC3100
End of banner message from server




Code:
Mem: 111324K used, 402392K free, 432K shrd, 4376K buff, 15724K cached
CPU:  0.0% usr  0.0% sys  0.0% nic 64.7% idle  0.0% io  0.0% irq 35.2% sirq
Load average: 4.79 1.80 0.95 4/119 16031
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 2298     1 root     R     1636  0.3   1 32.7 nas -P /tmp/nas.wl0lan.pid -H 34954 -l br0 -i eth1 -A -m 128 -k Gall3ry -s Peanuts2.4 -w 4 -g 3600



Mem: 112240K used, 401476K free, 432K shrd, 4376K buff, 15852K cached
CPU:  0.6% usr  6.2% sys  0.0% nic 55.1% idle  0.0% io  0.0% irq 37.9% sirq
Load average: 5.14 1.97 1.01 3/119 16074
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
  734     1 root     D     1628  0.3   0 21.7 watchdog
 2298     1 root     D     1636  0.3   0 20.1 nas -P /tmp/nas.wl0lan.pid -H 34954 -l br0 -i eth1 -A -m 128 -k Gall3ry -s Peanuts2.4 -w 4 -g 3600




Mem: 105928K used, 407788K free, 432K shrd, 4464K buff, 15968K cached
CPU: 50.0% usr  0.0% sys  0.0% nic  0.0% idle  0.0% io  0.0% irq 50.0% sirq
Load average: 6.14 2.29 1.13 6/121 16111
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 2298     1 root     R     1636  0.3   1 45.7 nas -P /tmp/nas.wl0lan.pid -H 34954 -l br0 -i eth1 -A -m 128 -k Gall3ry -s Peanuts2.4 -w 4 -g 3600
16110  1120 root     R     1336  0.2   0 45.7 dropbear -b /tmp/loginprompt -r /tmp/root/.ssh/ssh_host_rsa_key -p 50505





Mem: 106916K used, 406800K free, 432K shrd, 4464K buff, 15984K cached
CPU:  5.0% usr 11.6% sys  0.0% nic 56.1% idle  0.0% io  0.0% irq 27.1% sirq
Load average: 7.19 2.64 1.25 5/120 16122
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 2298     1 root     R     1636  0.3   1 43.4 nas -P /tmp/nas.wl0lan.pid -H 34954 -l br0 -i eth1 -A -m 128 -k Gall3ry -s Peanuts2.4 -w 4 -g 3600


Mem: 99524K used, 414192K free, 408K shrd, 712K buff, 15160K cached
CPU:  0.0% usr  6.6% sys  0.0% nic 63.8% idle  0.0% io  0.0% irq 29.3% sirq
Load average: 4.10 1.82 0.81 5/121 2909
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
  735     1 root     R     1628  0.3   0 35.8 watchdog
 2909  2908 root     R     1512  0.2   1  0.0 top -d10 -bHm -n2
 2389     1 root     S     1360  0.2   1  0.0 resetbutton


Every 11s: top -d10 -bHm -n2                                                                                                                          2022-11-13 00:00:37

Mem: 99160K used, 414556K free, 400K shrd, 712K buff, 15148K cached
CPU:  0.0% usr  5.8% sys  0.0% nic 58.8% idle  0.0% io  0.0% irq 35.2% sirq
Load average: 3.33 1.63 0.74 5/121 2909
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
  735     1 root     R     1628  0.3   0 41.1 watchdog
 2693     1 root     D     7540  1.4   1  0.0 ntpd -c /opt/etc/ntp.conf -f /opt/var/spool/ntp/ntp.drift -s /opt/var/spool/ntp -k /opt/etc/ntp -l /opt/var/spool/ntp/ntp.log -g


Mem: 112708K used, 401008K free, 432K shrd, 4376K buff, 15816K cached
CPU:  0.1% usr  0.4% sys  0.0% nic 63.3% idle  0.0% io  0.0% irq 36.0% sirq
Load average: 3.56 1.47 0.83 3/120 16030
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 1780     1 root     D     1292  0.2   0 35.1 radvd -C /tmp/radvd.conf
  160     2 root     RW       0  0.0   1  0.8 [kworker/1:1]
 1063     1 root     S     1496  0.2   0  0.1 syslogd -Z -L -R 192.168.0.14





Code:
[root@linuxhost01 zabbix]# ping -D 192.168.0.6 | tee -a result-of-ping.log
PING 192.168.0.6 (192.168.0.6) 56(84) bytes of data.
[1668315947.130004] 64 bytes from 192.168.0.6: icmp_seq=1 ttl=64 time=9135 ms
[1668315947.134973] 64 bytes from 192.168.0.6: icmp_seq=2 ttl=64 time=8140 ms
[1668315947.135054] 64 bytes from 192.168.0.6: icmp_seq=3 ttl=64 time=7140 ms
[1668315947.135066] 64 bytes from 192.168.0.6: icmp_seq=4 ttl=64 time=6140 ms
[1668315947.137388] 64 bytes from 192.168.0.6: icmp_seq=5 ttl=64 time=5142 ms
[1668315948.036113] 64 bytes from 192.168.0.6: icmp_seq=11 ttl=64 time=40.6 ms
[1668315949.044086] 64 bytes from 192.168.0.6: icmp_seq=12 ttl=64 time=47.7 ms
[1668315950.016615] 64 bytes from 192.168.0.6: icmp_seq=13 ttl=64 time=19.3 ms
[1668315951.021973] 64 bytes from 192.168.0.6: icmp_seq=14 ttl=64 time=24.0 ms






[1668315015.591975] 64 bytes from 192.168.0.6: icmp_seq=2985 ttl=64 time=9.00 ms
[1668315016.605565] 64 bytes from 192.168.0.6: icmp_seq=2986 ttl=64 time=21.3 ms
[1668315018.014138] 64 bytes from 192.168.0.6: icmp_seq=2987 ttl=64 time=428 ms
[1668315018.671860] 64 bytes from 192.168.0.6: icmp_seq=2988 ttl=64 time=85.8 ms
[1668315019.600658] 64 bytes from 192.168.0.6: icmp_seq=2989 ttl=64 time=13.6 ms
[1668315020.591057] 64 bytes from 192.168.0.6: icmp_seq=2990 ttl=64 time=2.16 ms
[1668315021.599760] 64 bytes from 192.168.0.6: icmp_seq=2991 ttl=64 time=9.47 ms
[1668315022.701964] 64 bytes from 192.168.0.6: icmp_seq=2992 ttl=64 time=109 ms
[1668315023.910556] 64 bytes from 192.168.0.6: icmp_seq=2993 ttl=64 time=317 ms
[1668315024.596295] 64 bytes from 192.168.0.6: icmp_seq=2994 ttl=64 time=1.79 ms
[1668315025.608961] 64 bytes from 192.168.0.6: icmp_seq=2995 ttl=64 time=12.3 ms
[1668315026.612617] 64 bytes from 192.168.0.6: icmp_seq=2996 ttl=64 time=14.4 ms
[1668315027.996485] 64 bytes from 192.168.0.6: icmp_seq=2997 ttl=64 time=396 ms
[1668315057.607704] From 192.168.0.104 icmp_seq=3024 Destination Host Unreachable
[1668315057.607796] From 192.168.0.104 icmp_seq=3025 Destination Host Unreachable
[1668315057.607808] From 192.168.0.104 icmp_seq=3026 Destination Host Unreachable
[1668315057.607816] From 192.168.0.104 icmp_seq=3027 Destination Host Unreachable
[1668315061.609706] From 192.168.0.104 icmp_seq=3028 Destination Host Unreachable
[1668315061.609784] From 192.168.0.104 icmp_seq=3029 Destination Host Unreachable
[1668315061.609795] From 192.168.0.104 icmp_seq=3030 Destination Host Unreachable
[1668315061.609803] From 192.168.0.104 icmp_seq=3031 Destination Host Unreachable
[1668315065.609771] From 192.168.0.104 icmp_seq=3032 Destination Host Unreachable
[1668315065.609845] From 192.168.0.104 icmp_seq=3033 Destination Host Unreachable
[1668315065.609864] From 192.168.0.104 icmp_seq=3034 Destination Host Unreachable
[1668315065.609879] From 192.168.0.104 icmp_seq=3035 Destination Host Unreachable
[1668315069.609706] From 192.168.0.104 icmp_seq=3036 Destination Host Unreachable
[1668315069.609777] From 192.168.0.104 icmp_seq=3037 Destination Host Unreachable
[1668315069.609788] From 192.168.0.104 icmp_seq=3038 Destination Host Unreachable
[1668315069.609797] From 192.168.0.104 icmp_seq=3039 Destination Host Unreachable
[1668315073.611677] From 192.168.0.104 icmp_seq=3040 Destination Host Unreachable
[1668315073.611751] From 192.168.0.104 icmp_seq=3041 Destination Host Unreachable
[1668315073.611763] From 192.168.0.104 icmp_seq=3042 Destination Host Unreachable
[1668315073.611771] From 192.168.0.104 icmp_seq=3043 Destination Host Unreachable
[1668315077.613749] From 192.168.0.104 icmp_seq=3044 Destination Host Unreachable
[1668315077.613855] From 192.168.0.104 icmp_seq=3045 Destination Host Unreachable
[1668315077.613879] From 192.168.0.104 icmp_seq=3046 Destination Host Unreachable
[1668315077.613891] From 192.168.0.104 icmp_seq=3047 Destination Host Unreachable
[1668315081.613760] From 192.168.0.104 icmp_seq=3048 Destination Host Unreachable
[1668315081.613842] From 192.168.0.104 icmp_seq=3049 Destination Host Unreachable
[1668315081.613853] From 192.168.0.104 icmp_seq=3050 Destination Host Unreachable
[1668315081.613862] From 192.168.0.104 icmp_seq=3051 Destination Host Unreachable
[1668315085.613742] From 192.168.0.104 icmp_seq=3052 Destination Host Unreachable
[1668315085.613819] From 192.168.0.104 icmp_seq=3053 Destination Host Unreachable
[1668315085.613830] From 192.168.0.104 icmp_seq=3054 Destination Host Unreachable
[1668315085.613838] From 192.168.0.104 icmp_seq=3055 Destination Host Unreachable
[1668315089.615740] From 192.168.0.104 icmp_seq=3056 Destination Host Unreachable
[1668315089.615805] From 192.168.0.104 icmp_seq=3057 Destination Host Unreachable
[1668315089.615816] From 192.168.0.104 icmp_seq=3058 Destination Host Unreachable
[1668315089.615824] From 192.168.0.104 icmp_seq=3059 Destination Host Unreachable
[1668315093.617709] From 192.168.0.104 icmp_seq=3060 Destination Host Unreachable
[1668315093.617787] From 192.168.0.104 icmp_seq=3061 Destination Host Unreachable
[1668315093.617798] From 192.168.0.104 icmp_seq=3062 Destination Host Unreachable
[1668315093.617806] From 192.168.0.104 icmp_seq=3063 Destination Host Unreachable
[1668315097.617766] From 192.168.0.104 icmp_seq=3064 Destination Host Unreachable
[1668315097.617866] From 192.168.0.104 icmp_seq=3065 Destination Host Unreachable
[1668315097.617886] From 192.168.0.104 icmp_seq=3066 Destination Host Unreachable
[1668315097.617900] From 192.168.0.104 icmp_seq=3067 Destination Host Unreachable
[1668315101.617689] From 192.168.0.104 icmp_seq=3068 Destination Host Unreachable
[1668315101.617759] From 192.168.0.104 icmp_seq=3069 Destination Host Unreachable
[1668315101.617771] From 192.168.0.104 icmp_seq=3070 Destination Host Unreachable
[1668315101.617779] From 192.168.0.104 icmp_seq=3071 Destination Host Unreachable
[1668315105.619682] From 192.168.0.104 icmp_seq=3072 Destination Host Unreachable
[1668315105.619771] From 192.168.0.104 icmp_seq=3073 Destination Host Unreachable
[1668315105.619791] From 192.168.0.104 icmp_seq=3074 Destination Host Unreachable
[1668315105.619801] From 192.168.0.104 icmp_seq=3075 Destination Host Unreachable
[1668315109.621705] From 192.168.0.104 icmp_seq=3076 Destination Host Unreachable
[1668315109.621774] From 192.168.0.104 icmp_seq=3077 Destination Host Unreachable
[1668315109.621785] From 192.168.0.104 icmp_seq=3078 Destination Host Unreachable
[1668315109.621793] From 192.168.0.104 icmp_seq=3079 Destination Host Unreachable
[1668315113.621688] From 192.168.0.104 icmp_seq=3080 Destination Host Unreachable
[1668315113.621747] From 192.168.0.104 icmp_seq=3081 Destination Host Unreachable
[1668315113.621758] From 192.168.0.104 icmp_seq=3082 Destination Host Unreachable
[1668315113.621766] From 192.168.0.104 icmp_seq=3083 Destination Host Unreachable
[1668315117.621765] From 192.168.0.104 icmp_seq=3084 Destination Host Unreachable
[1668315117.621809] From 192.168.0.104 icmp_seq=3085 Destination Host Unreachable
[1668315117.621819] From 192.168.0.104 icmp_seq=3086 Destination Host Unreachable
[1668315117.621828] From 192.168.0.104 icmp_seq=3087 Destination Host Unreachable
[1668315121.623698] From 192.168.0.104 icmp_seq=3088 Destination Host Unreachable
[1668315121.623755] From 192.168.0.104 icmp_seq=3089 Destination Host Unreachable
[1668315121.623765] From 192.168.0.104 icmp_seq=3090 Destination Host Unreachable
[1668315121.623774] From 192.168.0.104 icmp_seq=3091 Destination Host Unreachable
[1668315125.625689] From 192.168.0.104 icmp_seq=3092 Destination Host Unreachable
[1668315125.626881] From 192.168.0.104 icmp_seq=3093 Destination Host Unreachable
[1668315125.626903] From 192.168.0.104 icmp_seq=3094 Destination Host Unreachable
[1668315125.626920] From 192.168.0.104 icmp_seq=3095 Destination Host Unreachable
[1668315241.629802] From 192.168.0.104 icmp_seq=3208 Destination Host Unreachable
[1668315241.630905] From 192.168.0.104 icmp_seq=3209 Destination Host Unreachable
[1668315241.630919] From 192.168.0.104 icmp_seq=3210 Destination Host Unreachable
[1668315241.630929] From 192.168.0.104 icmp_seq=3211 Destination Host Unreachable
[1668315245.631832] From 192.168.0.104 icmp_seq=3212 Destination Host Unreachable
[1668315245.631878] From 192.168.0.104 icmp_seq=3213 Destination Host Unreachable
[1668315245.631893] From 192.168.0.104 icmp_seq=3214 Destination Host Unreachable
[1668315245.631909] From 192.168.0.104 icmp_seq=3215 Destination Host Unreachable
[1668315248.637844] From 192.168.0.104 icmp_seq=3216 Destination Host Unreachable
[1668315248.637919] From 192.168.0.104 icmp_seq=3217 Destination Host Unreachable
[1668315248.637934] From 192.168.0.104 icmp_seq=3218 Destination Host Unreachable
[1668315252.633708] From 192.168.0.104 icmp_seq=3219 Destination Host Unreachable
[1668315252.633794] From 192.168.0.104 icmp_seq=3220 Destination Host Unreachable
[1668315252.633807] From 192.168.0.104 icmp_seq=3221 Destination Host Unreachable
[1668315252.633815] From 192.168.0.104 icmp_seq=3222 Destination Host Unreachable
[1668315256.633704] From 192.168.0.104 icmp_seq=3223 Destination Host Unreachable
[1668315256.633781] From 192.168.0.104 icmp_seq=3224 Destination Host Unreachable
[1668315256.633792] From 192.168.0.104 icmp_seq=3225 Destination Host Unreachable
[1668315256.633800] From 192.168.0.104 icmp_seq=3226 Destination Host Unreachable
[1668315260.635701] From 192.168.0.104 icmp_seq=3227 Destination Host Unreachable
[1668315260.635768] From 192.168.0.104 icmp_seq=3228 Destination Host Unreachable
[1668315260.635781] From 192.168.0.104 icmp_seq=3229 Destination Host Unreachable
[1668315260.635789] From 192.168.0.104 icmp_seq=3230 Destination Host Unreachable
[1668315264.637655] From 192.168.0.104 icmp_seq=3231 Destination Host Unreachable
[1668315264.637734] From 192.168.0.104 icmp_seq=3232 Destination Host Unreachable
[1668315264.637744] From 192.168.0.104 icmp_seq=3233 Destination Host Unreachable
[1668315264.637753] From 192.168.0.104 icmp_seq=3234 Destination Host Unreachable
[1668315268.637712] From 192.168.0.104 icmp_seq=3235 Destination Host Unreachable
[1668315268.637819] From 192.168.0.104 icmp_seq=3236 Destination Host Unreachable
[1668315268.637831] From 192.168.0.104 icmp_seq=3237 Destination Host Unreachable
[1668315268.637839] From 192.168.0.104 icmp_seq=3238 Destination Host Unreachable
[1668315272.637684] From 192.168.0.104 icmp_seq=3239 Destination Host Unreachable
[1668315272.637764] From 192.168.0.104 icmp_seq=3240 Destination Host Unreachable
[1668315272.637776] From 192.168.0.104 icmp_seq=3241 Destination Host Unreachable
[1668315272.637784] From 192.168.0.104 icmp_seq=3242 Destination Host Unreachable
[1668315276.639701] From 192.168.0.104 icmp_seq=3243 Destination Host Unreachable
[1668315276.639782] From 192.168.0.104 icmp_seq=3244 Destination Host Unreachable
[1668315276.639793] From 192.168.0.104 icmp_seq=3245 Destination Host Unreachable
[1668315276.639802] From 192.168.0.104 icmp_seq=3246 Destination Host Unreachable
[1668315280.641695] From 192.168.0.104 icmp_seq=3247 Destination Host Unreachable
[1668315280.641794] From 192.168.0.104 icmp_seq=3248 Destination Host Unreachable
[1668315280.641814] From 192.168.0.104 icmp_seq=3249 Destination Host Unreachable
[1668315280.641831] From 192.168.0.104 icmp_seq=3250 Destination Host Unreachable
[1668315284.641702] From 192.168.0.104 icmp_seq=3251 Destination Host Unreachable
[1668315284.641781] From 192.168.0.104 icmp_seq=3252 Destination Host Unreachable
[1668315284.641793] From 192.168.0.104 icmp_seq=3253 Destination Host Unreachable
[1668315284.641802] From 192.168.0.104 icmp_seq=3254 Destination Host Unreachable
[1668315288.641717] From 192.168.0.104 icmp_seq=3255 Destination Host Unreachable
[1668315288.641798] From 192.168.0.104 icmp_seq=3256 Destination Host Unreachable
[1668315288.641809] From 192.168.0.104 icmp_seq=3257 Destination Host Unreachable
[1668315288.641818] From 192.168.0.104 icmp_seq=3258 Destination Host Unreachable
[1668315292.643703] From 192.168.0.104 icmp_seq=3259 Destination Host Unreachable
[1668315292.643796] From 192.168.0.104 icmp_seq=3260 Destination Host Unreachable
[1668315292.643810] From 192.168.0.104 icmp_seq=3261 Destination Host Unreachable
[1668315292.643822] From 192.168.0.104 icmp_seq=3262 Destination Host Unreachable
[1668315296.645723] From 192.168.0.104 icmp_seq=3263 Destination Host Unreachable
[1668315296.645820] From 192.168.0.104 icmp_seq=3264 Destination Host Unreachable
[1668315296.645842] From 192.168.0.104 icmp_seq=3265 Destination Host Unreachable
[1668315296.645857] From 192.168.0.104 icmp_seq=3266 Destination Host Unreachable
[1668315300.645710] From 192.168.0.104 icmp_seq=3267 Destination Host Unreachable
[1668315300.645791] From 192.168.0.104 icmp_seq=3268 Destination Host Unreachable
[1668315300.645802] From 192.168.0.104 icmp_seq=3269 Destination Host Unreachable
[1668315300.645810] From 192.168.0.104 icmp_seq=3270 Destination Host Unreachable
[1668315304.645772] From 192.168.0.104 icmp_seq=3271 Destination Host Unreachable
[1668315304.645840] From 192.168.0.104 icmp_seq=3272 Destination Host Unreachable
[1668315304.645851] From 192.168.0.104 icmp_seq=3273 Destination Host Unreachable
[1668315304.645859] From 192.168.0.104 icmp_seq=3274 Destination Host Unreachable
[1668315308.647695] From 192.168.0.104 icmp_seq=3275 Destination Host Unreachable
[1668315308.647765] From 192.168.0.104 icmp_seq=3276 Destination Host Unreachable
[1668315308.647776] From 192.168.0.104 icmp_seq=3277 Destination Host Unreachable
[1668315308.647785] From 192.168.0.104 icmp_seq=3278 Destination Host Unreachable
[1668315312.649693] From 192.168.0.104 icmp_seq=3279 Destination Host Unreachable
[1668315312.649770] From 192.168.0.104 icmp_seq=3280 Destination Host Unreachable
[1668315312.649782] From 192.168.0.104 icmp_seq=3281 Destination Host Unreachable
[1668315312.649791] From 192.168.0.104 icmp_seq=3282 Destination Host Unreachable
[1668315316.649680] From 192.168.0.104 icmp_seq=3283 Destination Host Unreachable
[1668315316.649756] From 192.168.0.104 icmp_seq=3284 Destination Host Unreachable
[1668315316.649768] From 192.168.0.104 icmp_seq=3285 Destination Host Unreachable
[1668315316.649776] From 192.168.0.104 icmp_seq=3286 Destination Host Unreachable
[1668315320.651698] From 192.168.0.104 icmp_seq=3287 Destination Host Unreachable
[1668315320.651777] From 192.168.0.104 icmp_seq=3288 Destination Host Unreachable
[1668315320.651787] From 192.168.0.104 icmp_seq=3289 Destination Host Unreachable
[1668315320.651796] From 192.168.0.104 icmp_seq=3290 Destination Host Unreachable
[1668315324.653680] From 192.168.0.104 icmp_seq=3291 Destination Host Unreachable
[1668315324.653757] From 192.168.0.104 icmp_seq=3292 Destination Host Unreachable
[1668315324.653769] From 192.168.0.104 icmp_seq=3293 Destination Host Unreachable
[1668315324.653778] From 192.168.0.104 icmp_seq=3294 Destination Host Unreachable
[1668315328.653733] From 192.168.0.104 icmp_seq=3295 Destination Host Unreachable
[1668315328.653848] From 192.168.0.104 icmp_seq=3296 Destination Host Unreachable
[1668315328.653859] From 192.168.0.104 icmp_seq=3297 Destination Host Unreachable
[1668315328.653868] From 192.168.0.104 icmp_seq=3298 Destination Host Unreachable
[1668315331.341002] 64 bytes from 192.168.0.6: icmp_seq=3299 ttl=64 time=1691 ms
[1668315331.341079] 64 bytes from 192.168.0.6: icmp_seq=3300 ttl=64 time=692 ms
[1668315332.450865] 64 bytes from 192.168.0.6: icmp_seq=3301 ttl=64 time=802 ms
[1668315335.674600] 64 bytes from 192.168.0.6: icmp_seq=3302 ttl=64 time=3026 ms
[1668315336.502411] 64 bytes from 192.168.0.6: icmp_seq=3303 ttl=64 time=2854 ms
[1668315338.768113] 64 bytes from 192.168.0.6: icmp_seq=3304 ttl=64 time=4120 ms
[1668315339.734762] 64 bytes from 192.168.0.6: icmp_seq=3305 ttl=64 time=4086 ms
[1668315339.745168] 64 bytes from 192.168.0.6: icmp_seq=3306 ttl=64 time=3097 ms
[1668315339.747359] 64 bytes from 192.168.0.6: icmp_seq=3307 ttl=64 time=2099 ms
[1668315339.751064] 64 bytes from 192.168.0.6: icmp_seq=3308 ttl=64 time=1103 ms
[1668315339.763701] 64 bytes from 192.168.0.6: icmp_seq=3309 ttl=64 time=114 ms
[1668315351.448898] 64 bytes from 192.168.0.6: icmp_seq=3310 ttl=64 time=10798 ms
[1668315351.451353] 64 bytes from 192.168.0.6: icmp_seq=3311 ttl=64 time=9800 ms
[1668315351.454020] 64 bytes from 192.168.0.6: icmp_seq=3312 ttl=64 time=8803 ms
[1668315351.458220] 64 bytes from 192.168.0.6: icmp_seq=3313 ttl=64 time=7807 ms
[1668315351.464647] 64 bytes from 192.168.0.6: icmp_seq=3314 ttl=64 time=6814 ms
[1668315351.467092] 64 bytes from 192.168.0.6: icmp_seq=3315 ttl=64 time=5816 ms
[1668315351.468977] 64 bytes from 192.168.0.6: icmp_seq=3316 ttl=64 time=4818 ms

_________________
Cheers, TK
------------------------
Sponsor
Hapi12021
DD-WRT User


Joined: 22 Jul 2021
Posts: 55

PostPosted: Sun Nov 13, 2022 23:06    Post subject: Reply with quote
Software interrupt requests (sIRQ) happen when the kernel is busy handling devices or pseudo devices.

My guess is that somehow you are inspecting the traffic on your internal network with the firewall or another packet filter, when it should be disabled for a strictly internal-only AP, or something like using RIP where a static route would be necessary, instead.

You could also have a bridging loop on your network and tripping STP, if enabled, or getting an old-school packet storm.

If radvd is chewing up CPU then look to devices that aren’t behaving properly on IPv6 or possibly having inadvertently blocked some core IPv6 neighbor protocol or port.
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Fri Nov 18, 2022 4:50    Post subject: Reply with quote
Thank you very much for the feedback. This is an external facing AP hence have been running a F/W on this unit. STP is disabled. Just switched over to CTF for the Shortcut Forwarding Engine but that had negligible effect. Do have IPV6 enabled however.

Trying to capture some tcpdump results at the time of the high SIRQ usages when router also reboots from time to time.

Captured additional logs. The time of 04:14 GMT is when the router hits 100% SIRQ:

Code:

Nov 18 04:06:06 dd-wrt-inet.mds.xyz kernel: [ 8395.188042] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:06:06 dd-wrt-inet.mds.xyz kernel: [ 8395.188050] jffs2: read_cache_page() returned error: -5
Nov 18 04:06:06 dd-wrt-inet.mds.xyz kernel: [ 8395.188055] jffs2: Error garbage collecting node at 058c06f0!
.
.
.
.
.
.
.
.
Nov 18 04:10:10 dd-wrt-inet.mds.xyz radvd[6889]: our AdvManagedFlag on br0 doesn't agree with fe80::ba27:ebff:fe1c:558d
Nov 18 04:10:10 dd-wrt-inet.mds.xyz radvd[6889]: our AdvLinkMTU on br0 doesn't agree with fe80::ba27:ebff:fe1c:558d (ours 1452, others 1500)
Nov 18 04:10:17 dd-wrt-inet.mds.xyz kernel: [ 8646.391240] JFFS2 compression type 0x06 not available.
Nov 18 04:10:17 dd-wrt-inet.mds.xyz kernel: [ 8646.396443] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:10:17 dd-wrt-inet.mds.xyz kernel: [ 8646.401631] jffs2: read_cache_page() returned error: -5
Nov 18 04:10:17 dd-wrt-inet.mds.xyz kernel: [ 8646.406872] jffs2: Error garbage collecting node at 058c06f0!
Nov 17 23:10:31 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPREQUEST(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 17 23:10:31 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPACK(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 18 04:10:31 dd-wrt-inet.mds.xyz kernel: [ 8660.568375] JFFS2 compression type 0x06 not available.
Nov 18 04:10:31 dd-wrt-inet.mds.xyz kernel: [ 8660.573532] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:10:31 dd-wrt-inet.mds.xyz kernel: [ 8660.578791] jffs2: read_cache_page() returned error: -5
Nov 18 04:10:31 dd-wrt-inet.mds.xyz kernel: [ 8660.584187] jffs2: Error garbage collecting node at 058c06f0!
Nov 17 23:10:32 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPREQUEST(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 17 23:10:32 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPACK(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 18 04:10:32 dd-wrt-inet.mds.xyz kernel: [ 8661.793593] JFFS2 compression type 0x06 not available.
Nov 18 04:10:32 dd-wrt-inet.mds.xyz kernel: [ 8661.793596] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:10:32 dd-wrt-inet.mds.xyz kernel: [ 8661.793605] jffs2: read_cache_page() returned error: -5
Nov 18 04:10:32 dd-wrt-inet.mds.xyz kernel: [ 8661.793610] jffs2: Error garbage collecting node at 058c06f0!
Nov 18 04:11:10 dd-wrt-inet.mds.xyz kernel: [ 8699.175132] dhd_msgbuf_wait_ioctl_cmplt: resumed on timeout rxcnt_timeout 1
Nov 18 04:11:10 dd-wrt-inet.mds.xyz kernel: [ 8699.175134] dhd_msgbuf_wait_ioctl_cmplt: timeout > MAX_CNTL_TX_TIMEOUT
Nov 18 04:11:10 dd-wrt-inet.mds.xyz kernel: [ 8699.587016] dhd_prot_ioctack_process: received ioctl ACK with state 01
Nov 18 04:11:10 dd-wrt-inet.mds.xyz kernel: [ 8699.587020] dhd_prot_ioctcmplt_process: received ioctl response with state 00
Nov 18 04:11:13 dd-wrt-inet.mds.xyz kernel: [ 8702.667043] dhd_msgbuf_wait_ioctl_cmplt: resumed on timeout rxcnt_timeout 2
Nov 18 04:11:28 dd-wrt-inet.mds.xyz kernel: [ 8717.351319] dhd_msgbuf_wait_ioctl_cmplt: timeout > MAX_CNTL_TX_TIMEOUT
Nov 18 04:14:49 dd-wrt-inet.mds.xyz kernel: [ 8918.079685] dhd_msgbuf_wait_ioctl_cmplt: resumed on timeout rxcnt_timeout 2
Nov 18 04:14:49 dd-wrt-inet.mds.xyz kernel: [ 8918.086671] dhd_msgbuf_wait_ioctl_cmplt: timeout > MAX_CNTL_TX_TIMEOUT
Nov 18 04:14:56 dd-wrt-inet.mds.xyz kernel: [ 8923.994957] dhd_msgbuf_wait_ioctl_cmplt: resumed on timeout rxcnt_timeout 1
Nov 18 04:14:56 dd-wrt-inet.mds.xyz kernel: [ 8923.994959] dhd_msgbuf_wait_ioctl_cmplt: timeout > MAX_CNTL_TX_TIMEOUT
Nov 18 04:14:56 dd-wrt-inet.mds.xyz kernel: [ 8925.817100] dhd_prot_ioctack_process: received ioctl ACK with state 01
Nov 18 04:14:56 dd-wrt-inet.mds.xyz kernel: [ 8925.823657] dhd_prot_ioctcmplt_process: received ioctl response with state 00
Nov 18 04:14:57 dd-wrt-inet.mds.xyz kernel: [ 8927.021387] dhd_msgbuf_wait_ioctl_cmplt: resumed on timeout rxcnt_timeout 2
.
.
.
.
.
.
.
.
.
.
.

Nov 18 04:15:04 dd-wrt-inet.mds.xyz kernel: [ 8933.803369] dhd_msgbuf_wait_ioctl_cmplt: timeout > MAX_CNTL_TX_TIMEOUT
Nov 18 04:15:20 dd-wrt-inet.mds.xyz kernel: [ 8949.107652] JFFS2 compression type 0x06 not available.
Nov 18 04:15:20 dd-wrt-inet.mds.xyz kernel: [ 8949.112815] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:15:20 dd-wrt-inet.mds.xyz kernel: [ 8949.118025] jffs2: read_cache_page() returned error: -5
Nov 18 04:15:20 dd-wrt-inet.mds.xyz kernel: [ 8949.123290] jffs2: Error garbage collecting node at 058c06f0!
Nov 18 04:18:52 dd-wrt-inet.mds.xyz radvd[6889]: our AdvManagedFlag on br0 doesn't agree with fe80::ba27:ebff:fe1c:558d
Nov 18 04:18:52 dd-wrt-inet.mds.xyz radvd[6889]: our AdvLinkMTU on br0 doesn't agree with fe80::ba27:ebff:fe1c:558d (ours 1452, others 1500)
Nov 17 23:19:31 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPREQUEST(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 17 23:19:31 dd-wrt-inet.mds.xyz dnsmasq-dhcp[6879]: DHCPACK(br0) 192.168.0.209 fe:c0:ab:09:55:93
Nov 18 04:19:31 dd-wrt-inet.mds.xyz kernel: [ 9200.577319] JFFS2 compression type 0x06 not available.
Nov 18 04:19:31 dd-wrt-inet.mds.xyz kernel: [ 9200.582509] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:19:31 dd-wrt-inet.mds.xyz kernel: [ 9200.587750] jffs2: read_cache_page() returned error: -5
Nov 18 04:19:31 dd-wrt-inet.mds.xyz kernel: [ 9200.593124] jffs2: Error garbage collecting node at 058c06f0!
.
.
.
.
.
.
Nov 18 04:20:19 dd-wrt-inet.mds.xyz kernel: [ 9248.983276] JFFS2 compression type 0x06 not available.
Nov 18 04:20:19 dd-wrt-inet.mds.xyz kernel: [ 9248.983279] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:20:19 dd-wrt-inet.mds.xyz kernel: [ 9248.983287] jffs2: read_cache_page() returned error: -5
Nov 18 04:20:19 dd-wrt-inet.mds.xyz kernel: [ 9248.983291] jffs2: Error garbage collecting node at 058c06f0!
.




Code:

Mem: 116784K used, 396932K free, 432K shrd, 84K buff, 13164K cached
CPU0:  0.0% usr  0.0% sys  0.0% nic  0.0% idle  0.0% io  0.0% irq  100% sirq
CPU1:  0.0% usr  2.5% sys  0.0% nic 96.0% idle  0.0% io  0.0% irq  1.2% sirq
Load average: 3.53 1.16 0.49 4/112 9210
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 1068     1 root     R     1496  0.2   0 35.9 syslogd -Z -L -R 192.168.0.14
 2866     2 root     SW       0  0.0   1  0.6 [kworker/1:0]
 6865     1 root     S     1524  0.3   1  0.2 watchquagga -dz -r %s -d zebra ospfd bgpd ripd
  206     2 root     SW       0  0.0   1  0.1 [kswapd0]
 8677  2731 root     R     1520  0.3   1  0.0 top -d 10



Saving settings yielded the following, not yet sure if any of these messages is a problem or not:

Code:

Nov 18 04:45:18 dd-wrt-inet.mds.xyz kernel: [10747.706665] JFFS2 compression type 0x06 not available.
Nov 18 04:45:18 dd-wrt-inet.mds.xyz kernel: [10747.711826] jffs2: Error: jffs2_decompress returned -5
Nov 18 04:45:18 dd-wrt-inet.mds.xyz kernel: [10747.717065] jffs2: read_cache_page() returned error: -5
Nov 18 04:45:18 dd-wrt-inet.mds.xyz kernel: [10747.722729] jffs2: Error garbage collecting node at 058c06f0!
Nov 18 04:45:23 dd-wrt-inet.mds.xyz httpd[6916]: _evalpid:touch /tmp/hosts
Nov 18 04:45:32 dd-wrt-inet.mds.xyz httpd[6916]: _evalpid:touch /tmp/hosts
Nov 18 04:45:32 dd-wrt-inet.mds.xyz httpd[6916]: _evalpid:ledtool 1
Nov 18 04:45:32 dd-wrt-inet.mds.xyz httpd[6916]: _evalpid:async_commit
Nov 18 04:45:32 dd-wrt-inet.mds.xyz : _evalpid:startservice_f init_user -f
Nov 18 04:45:32 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service init_user start -f
Nov 18 04:45:32 dd-wrt-inet.mds.xyz : _evalpid:start_single_service
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service overclocking start
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service wan stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service nas stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [servicemanager] : waiting for services to finish (12)...
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service lan stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service bridging stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service ttraff stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service emf stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service vlantagging stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [nas] : daemon trying to stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service bridgesif stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service bonding stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service ipvs stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service radio_timer stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [ipvsadm] : ipvs daemon trying to stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:ifconfig br0 down
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [openvpnserver] : OpenVPN daemon (Server) trying to stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:vconfig set_name_type DEV_PLUS_VID_NO_PAD
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [ttraff] : traffic counter daemon trying to stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [bridge] : interface wl0.1 successfully deleted from bridge br0
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:brctl delif br0 wl0.1
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : [radio_timer] : daemon trying to stop
Nov 18 04:45:37 dd-wrt-inet.mds.xyz : _evalpid:rmmod bonding
Nov 18 04:46:28 dd-wrt-inet.mds.xyz process_monitor[12026]: _evalpid:stopservice ntpc -f
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:/sbin/service ntpc stop
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : [ntpclient] : Network Time Protocol client trying to stop
Nov 18 04:46:28 dd-wrt-inet.mds.xyz process_monitor[12026]: _evalpid:ntpclient 192.168.0.12
Nov 18 04:46:28 dd-wrt-inet.mds.xyz ntpclient[12060]: Connecting to 192.168.0.12 [192.168.0.12] ...
Nov 18 04:46:28 dd-wrt-inet.mds.xyz ntpclient[12060]: Time set from 192.168.0.12 [192.168.0.12].
Nov 18 04:46:28 dd-wrt-inet.mds.xyz process_monitor[12026]: Cyclic NTP Update success (servers 192.168.0.12)
Nov 18 04:46:28 dd-wrt-inet.mds.xyz process_monitor[12026]: Local timer delta is 0
Nov 18 04:46:28 dd-wrt-inet.mds.xyz process_monitor[12026]: _evalpid:filtersync
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_1
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -I lan2wan -j grp_1
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_2
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_3
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -I lan2wan -j grp_3
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_4
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_5
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_6
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_7
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_8
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_9
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_10
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_11
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:28 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_12
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_13
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_14
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_15
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_16
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_17
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_18
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_19
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_20
Nov 18 04:46:29 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock



Just set the MTU to Auto to get rid of at least one of the messages above. (Doubt it has much to do with the issue at the moment.)

_________________
Cheers, TK
------------------------
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Fri Nov 18, 2022 5:06    Post subject: Reply with quote
A few messages around shutdown:

Code:
Nov 17 08:58:33 dd-wrt-inet.mds.xyz process_monitor[2274]: Cyclic NTP Update success (servers 192.168.0.12)
Nov 17 08:58:33 dd-wrt-inet.mds.xyz process_monitor[2274]: Local timer delta is 0
Nov 17 08:58:33 dd-wrt-inet.mds.xyz process_monitor[2274]: _evalpid:filtersync
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:rmmod fast-classifier
Nov 17 08:58:33 dd-wrt-inet.mds.xyz kernel: [ 3640.703224] fast-classifier: shutting down
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe-ipv6
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : [sfe] : shortcut forwarding engine successfully stopped
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_1
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_2
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:iptables -I lan2wan -j grp_2
Nov 17 08:58:33 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock



Code:

Nov 17 10:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: Cyclic NTP Update success (servers 192.168.0.12)
Nov 17 10:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: Local timer delta is 0
Nov 17 10:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: _evalpid:filtersync
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod fast-classifier
Nov 17 10:58:34 dd-wrt-inet.mds.xyz kernel: [10841.602519] fast-classifier: shutting down
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe-ipv6
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : [sfe] : shortcut forwarding engine successfully stopped
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_1
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_2
Nov 17 10:58:34 dd-wrt-inet.mds.xyz : _evalpid:iptables -I lan2wan -j grp_2



Code:

Nov 17 11:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: Cyclic NTP Update success (servers 192.168.0.12)
Nov 17 11:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: Local timer delta is 0
Nov 17 11:58:34 dd-wrt-inet.mds.xyz process_monitor[2274]: _evalpid:filtersync
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod fast-classifier
Nov 17 11:58:34 dd-wrt-inet.mds.xyz kernel: [14442.035049] fast-classifier: shutting down
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe-ipv6
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : [sfe] : shortcut forwarding engine successfully stopped
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_1
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:rm -f /tmp/.fw_lock
Nov 17 11:58:34 dd-wrt-inet.mds.xyz : _evalpid:iptables -D lan2wan -j grp_2



Code:
Nov 17 12:58:35 dd-wrt-inet.mds.xyz process_monitor[2274]: Cyclic NTP Update success (servers 192.168.0.12)
Nov 17 12:58:35 dd-wrt-inet.mds.xyz process_monitor[2274]: Local timer delta is 0
Nov 17 12:58:35 dd-wrt-inet.mds.xyz process_monitor[2274]: _evalpid:filtersync
Nov 17 12:58:35 dd-wrt-inet.mds.xyz : _evalpid:rmmod fast-classifier
Nov 17 12:58:35 dd-wrt-inet.mds.xyz kernel: [18042.542831] fast-classifier: shutting down
Nov 17 12:58:35 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe-ipv6
Nov 17 12:58:35 dd-wrt-inet.mds.xyz : _evalpid:rmmod shortcut-fe



and this repeats. Besides uptime, how to show last set of reboots of the router when 'last' command is not available and not installable via opkg?

Cheers,

_________________
Cheers, TK
------------------------
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Fri Nov 18, 2022 6:07    Post subject: Reply with quote
"or getting an old-school packet storm."

Possibly, based on some tcpdumps taken when sirq is high:

Code:
00:07:46.626528 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839985174:1839986622, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448
00:07:46.626876 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839986622:1839988070, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448
00:07:46.627224 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839988070:1839989518, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448
00:07:46.627570 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839989518:1839990966, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448
00:07:46.627938 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839990966:1839992414, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448
00:07:46.628286 IP 10.3.0.237.22 > 192.168.0.14.44990: Flags [.], seq 1839992414:1839993862, ack 790146, win 1432, options [nop,nop,TS val 4191449188 ecr 4251986240], length 1448



There's 10's of thousands of these at the time when SIRQ is high. Can't figure out why yet or where these come from. Not seeing anything on that one 10.3.0.237 host yet.

However, the high sirq and random reboots happen when browsing the web. Otherwise, everything is fine.

Cheers,

_________________
Cheers, TK
------------------------
Mile-Lile
DD-WRT Guru


Joined: 24 Feb 2013
Posts: 1574
Location: Belgrade

PostPosted: Fri Nov 18, 2022 6:41    Post subject: Reply with quote
I saw similar behaviour on my r8000 with brcmfmac (but not with dhd)...

can you post output of:

Code:
cat /proc/softirqs


Code:
cat /proc/interrupts


and screenshot of

Code:
htop


if you have problem with htop display just do:

Code:
export TERM=xterm



for compare?
egc
DD-WRT Guru


Joined: 18 Mar 2014
Posts: 10788
Location: Netherlands

PostPosted: Fri Nov 18, 2022 8:49    Post subject: Reply with quote
First follow @Hapi12021's and @Mile-Lile's advice

But other things to try, do not use SFE (shortcut forwarding engine on setup page) but if you need the bandwidth use CTF + FA, if you do not need the bandwidth do not use any acceleration.

Not sure what yo are running on the router but if you have a lot of applications running you might need a SWAP file

_________________
Routers:Netgear R7800, R7000, R6400v1, R6400v2, Linksys EA8500, EA6900 (XvortexCFE), E2000, E1200v1, WRT54GS v1.
Install guide R6400v2, R6700v3,XR300:https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=316399
Install guide R7800/XR500: https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=320614
Forum Guide Lines (important read):https://forum.dd-wrt.com/phpBB2/viewtopic.php?t=324087
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sat Nov 19, 2022 1:29    Post subject: Reply with quote
Code:

# cat /proc/softirqs
                    CPU0       CPU1
          HI:          1          0
       TIMER:    5633256    5548263
      NET_TX:         16         91
      NET_RX:    9289989    8462498
       BLOCK:       1002       1238
BLOCK_IOPOLL:          0          0
     TASKLET:   68077216    1476722
       SCHED:    5610483    5425056
     HRTIMER:          0          0
         RCU:    2752391    3115464



Code:

# cat /proc/interrupts
           CPU0       CPU1
 27:          6          0       GIC  27 Edge      mpcore_gtimer
 29:    5649168    5609717       GIC  29 Edge      twd
 32:          0          0       GIC  32 Edge      L2C
111:       6650        429       GIC 111 Edge      ehci_hcd:usb1, ohci_hcd:usb2
112:          1          0       GIC 112 Edge      xhci-hcd:usb3
117:      47073      26909       GIC 117 Edge      serial
163:     430427     666103       GIC 163 Edge      dhdpcie:0001:01:00.0
169:     318832     525678       GIC 169 Edge      dhdpcie:0002:01:00.0
181:   57747633          0       GIC 181 Edge      eth0
IPI0:         0          0       CPU wakeup interrupts
IPI1:         0          1       Timer broadcast interrupts
IPI2:   2333321    2689848       Rescheduling interrupts
IPI3:         0          0       Function call interrupts
IPI4:     61758    8400593       Single function call interrupts
IPI5:         0          0       CPU stop interrupts
IPI6:         0          0       IRQ work interrupts
IPI7:         0          0       completion interrupts
Err:          0


Attached htop with an average over 10 seconds.

My routers are set up with the following, which the exception of OpenVPN which only runs on the subject router here.

Operating Mode: vtysh OSPF BGP RIP router

Setup: 'vtysh OSPF BGP RIP router' Router (Internet Facing), Interconnected and working perfectly with OSPF on Cisco 3750G + Cisco WS-C4948, Works with 5 Separate VLAN's, VPN Server w/ AES-256, DNSMasq, JFFS, Entware, USB drive added, Syslog, SSH, Wireless NG 5G / 2.4G NA, iptables. Interconnected with other Asus RT-AC88U / RT-AC1300 routers w/ v3.0-r49392 FW also running OSPF, iptables, Zabbix Monitoring Agent etc (as per above) and similar config with the exception of OpenVPN not running on the internal routers.

There are ~10 VLAN's in the mix, some of which are defined on internal DD-WRT routers.



htop-d100-ddwrt.PNG
 Description:
 Filesize:  162.94 KB
 Viewed:  477 Time(s)

htop-d100-ddwrt.PNG



_________________
Cheers, TK
------------------------
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sat Nov 19, 2022 1:46    Post subject: Reply with quote
egc wrote:
First follow @Hapi12021's and @Mile-Lile's advice

But other things to try, do not use SFE (shortcut forwarding engine on setup page) but if you need the bandwidth use CTF + FA, if you do not need the bandwidth do not use any acceleration.

Not sure what yo are running on the router but if you have a lot of applications running you might need a SWAP file


Absolutely! Just posted to EGC and Hapi12021's comments.

Router hasn't rebooted 'spontaneously' today after a few small changes including using CTF though admittedly the activity on the network hasn't been under the same kind of load today as it was on previous weekdays. Will see over the weekend. MEM appears good. Lot's of free and inactive pages:

Code:
# cat /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  526045184 119873536 406171648   221184 13275136 17444864
Swap:        0        0        0
MemTotal:         513716 kB
MemFree:          396652 kB
MemShared:           216 kB
Buffers:           12964 kB
Cached:            17036 kB
SwapCached:            0 kB
Active:            27172 kB
Inactive:          14144 kB
MemAvailable:     335628 kB
Active(anon):      11456 kB
Inactive(anon):      216 kB
Active(file):      15716 kB
Inactive(file):    13928 kB
Unevictable:         132 kB
Mlocked:               0 kB
HighTotal:        393216 kB
HighFree:         363952 kB
LowTotal:         120500 kB
LowFree:           32700 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         11508 kB
Mapped:            10896 kB
Shmem:               216 kB
Slab:              12764 kB
SReclaimable:       1912 kB
SUnreclaim:        10852 kB
KernelStack:         952 kB
PageTables:          736 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      410972 kB
Committed_AS:      20484 kB
VmallocTotal:    1949696 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB


Sorry, didn't catch what 'FA' stands for here. Could you please elaborate? Otherwise, here's the few changes I've done:

- Set to CTF
- Changed the router reboot schedule and disabled another reboot (had two, disabled one, moved the other time)
- Set MTU to Auto
- Disabled all features except 'Explicit Beamforming' from Wifi config. Though did this several days ago with no effect. Doesn't appear enabling any of the Wireless settings make any difference for me, just mess things up.

SWAP file is possible, but haven't set one yet due to plenty of free memory. Have USB storage here w/ 28GB available for SWAP if truly needed.

Router's been up for 16+ hours so far today since the nightly scheduled reboot.

_________________
Cheers, TK
------------------------
Mile-Lile
DD-WRT Guru


Joined: 24 Feb 2013
Posts: 1574
Location: Belgrade

PostPosted: Sat Nov 19, 2022 7:33    Post subject: Reply with quote
FA is some kind of NAT acceleration and is brcm specific... On some router models you don't have option enable/disable (e.g. R8000) because it is already enabled (not sure) by default (it is usually on Index.asp at SFE section)... you can enable it with nvram variable but it will cause router to bootloop... outputs and screenshot you posted looks normal...

I have had disabled SFE, CTF&FA but still had crashes (brcmfmac)... Then when I disabled everything I use (ipv6, WG, QoS, NextDNS CLI) I didn't had crashes... I had 10 days router uptime... but as soon as I enable for example ipv6 after some time brcmfmac starts crashing... output of crash doesn't tell much... BS told me cause is unknown... and is very hard to find it... But I noticed similar symptoms like you do (during these 10 days of uptime)... CPU spikes goes to 2.5... with just basic functions... With normal build (non-experimental with dhd driver) I use bunch of apps on router, and router is rock solid... but wifi is much better on brcmfmac...
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sat Nov 19, 2022 20:17    Post subject: Reply with quote
When SFE is disabled, DNS resolution is toast, except for internal DNS resolution to internal DNS hosts.

For example can't nslookup any domain. There's no result. Guess this would explain why SIRQ is very high. All DNS requests are going through this service which would explain why simple browing resolution is slow and causes the router to reboot. Now I'm trying to find how to enable DNS resolution without enabling SFE or CTF. Even typing something like nslookup SITE EXTERNAL-DNS fails.

Thank you for the write up Mile-Lile. Flow Acceleration (FA) was already off:

# nvram show|grep -Ei ctf_fa_cap
size: 55203 bytes (75869 left)
ctf_fa_cap=0


Cheers,

_________________
Cheers, TK
------------------------
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sun Nov 20, 2022 0:24    Post subject: Reply with quote
Looks like these settings across my routers seeming improved things a bit. Time will tell:

Shortcut Forwarding Engine: CTF
STP: Disabled

MTU:    Manual    1452

# nvram show|grep -Ei ctf_fa_cap=1
size: 55223 bytes (75849 left)
ctf_fa_cap=1

NOTE: The last option may or may not be ideal for your router, depending if Flow Acceleration (FA) module is included in your setup.

https://microdevsys.com/wp/dd-wrt-network-performance-tuning/

_________________
Cheers, TK
------------------------
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Sun Nov 20, 2022 23:18    Post subject: Reply with quote
Looks like I'll need DNS caching on my DNS servers. (facepalm) In the span of 5 minutes, 4K DNS queries occurred. This doesn't look right nor would it be appropriate for the system to do this many DNS queries.

Code:
# grep -Ei "DPT=53" dd-wrt-inet.mds.xyz.log|wc -l
4155
#


Coincides with the router rebooting and latency like this:

Code:
bytes from 192.168.0.6: seq=56 ttl=64 time=0.375 ms
64 bytes from 192.168.0.6: seq=57 ttl=64 time=94.126 ms
64 bytes from 192.168.0.6: seq=58 ttl=64 time=321.408 ms
64 bytes from 192.168.0.6: seq=59 ttl=64 time=61859.628 ms
64 bytes from 192.168.0.6: seq=60 ttl=64 time=62368.570 ms
64 bytes from 192.168.0.6: seq=61 ttl=64 time=61361.560 ms
64 bytes from 192.168.0.6: seq=62 ttl=64 time=60359.392 ms
64 bytes from 192.168.0.6: seq=63 ttl=64 time=59357.949 ms
64 bytes from 192.168.0.6: seq=64 ttl=64 time=58355.773 ms
64 bytes from 192.168.0.6: seq=65 ttl=64 time=57349.340 ms
64 bytes from 192.168.0.6: seq=130 ttl=64 time=3331.460 ms
64 bytes from 192.168.0.6: seq=131 ttl=64 time=2322.112 ms
64 bytes from 192.168.0.6: seq=132 ttl=64 time=1312.591 ms
64 bytes from 192.168.0.6: seq=133 ttl=64 time=303.072 ms


Cheers,

_________________
Cheers, TK
------------------------
Alozaros
DD-WRT Guru


Joined: 16 Nov 2015
Posts: 5380
Location: UK, London, just across the river..

PostPosted: Sun Nov 20, 2022 23:52    Post subject: Reply with quote
hmm, looking at the thread and your heavy router use...do have USB as a storage as this is very CPU heavy if more than one client is using it at the same time...than you may need to increase the USB buffer size..may be..

also speaking of DNS cashing..yep its a good to have it, if there a many client at the same time...you can even set it to 2000-3000-5000 as default it 1500 (an this is lines(hits) not kb or bites) although 1500 is more than enough...

also to improve DNS if you use DNSmasq... you can increase the maximum number of concurrent DNS queries
the default value is 150, but you can set it to 300 or 500 if so...
add this line to advanced DNSmasq rules

dns-forward-max=500

for an edge routers with heavy use:
and lots of DNS requests you can fiddle with few dnsmasq options too
on heavy load, systems those are the settings i use in general

domain-needed
no-ping
no-resov
server=9.9.9.9
no-negcache
cache-size=3000
filterwin2k
dhcp-option=43,01:04:00:00:00:02
dns-loop-detect
quiet-dhcp

you can also fiddle with:
max-cache-ttl=
min-cache-ttl=
max-ttl=

https://dnsmasq.org/docs/dnsmasq-man.html

_________________
Atheros
TP-Link WR740Nv1 ---DD-WRT 50500 WAP
TP-Link WR1043NDv2 -DD-WRT 50841 Gateway,DNS,AP Isolation,Ad-Block,Firewall,Forced DNS,DoT,VPN,VLAN
TP-Link WR1043NDv2 -DD-WRT 50841 Gateway,DNS,Ad-Block,Firewall,Forced DNS,DoT,VPN,VLAN
TP-Link WR1043NDv2 -Gargoyle OS 1.13.0 AP,DNS,QoS,Quotas
Qualcomm-Atheros
Netgear R7800 --DD-WRT 50963 Gateway,DNS,AD-Block,AP&Net Isolation,VLAN's,Firewall,DoT,Vanilla
Netgear R9000 --DD-WRT 50927 Gateway,DNS,AD-Block,AP Isolation,Firewall,Forced DNS,DoT,2,4Ghz only,Vanilla
Broadcom
Netgear R7000 ---DD-WRT 50927 Gateway,DNS,AD-Block,Firewall,Forced DNS,VLAN's,DoT,VPN
NOT USING 5Ghz ANYWHERE
------------------------------------------------------
Stubby for DNS over TLS I DNSCrypt v2 by mac913
tkmds
DD-WRT User


Joined: 04 Sep 2011
Posts: 142
Location: SA

PostPosted: Mon Nov 21, 2022 5:36    Post subject: Reply with quote
Thanks @Alozaros!

I have DNSmasquerade disabled. Haven't been using it for some time. I rely on 6 internal DNS servers, 3x2 clusters. I've enabled caching on one of them, set it to 64MB with a 2 day expiry. That DNS server was set to 0KB w/ 1 day expiry, hence apparently disabled I guess. Page loading and lookup is way faster. So this helps. Will monitor.

Can DNSmasquerade help with external DNS caching in any way? Tried enabling it but it didn't work. I'll have to spend more time to figure out how to get DNSmasquerade working w/ DNS servers sitting on VM's on the internal network. Smile

I do have IPV6 enabled as well on the 4 DDWRT routers. I have all 4 with same FW version. Thinking this is contributing heavily but can't prove it yet. Disabled non OSPF services like BGP to reduce memory and CPU consumption which should help a bit.

For USG buffer size, thinking you mean "Drive Read-Ahead Buffer"? It's currently set to 256.

Regardless, on a secondary router, I ended up getting OOM with version 49392, which leads to reboots. Attached the logs of the OOM.

Can't pin point the culprit yet.

Cheers,



roma-oom.png
 Description:
 Filesize:  137.8 KB
 Viewed:  284 Time(s)

roma-oom.png



oom-killer-49392.txt
 Description:

Download
 Filename:  oom-killer-49392.txt
 Filesize:  242.87 KB
 Downloaded:  3 Time(s)


_________________
Cheers, TK
------------------------
Goto page 1, 2  Next Display posts from previous:    Page 1 of 2
Post new topic   Reply to topic    DD-WRT Forum Index -> Broadcom SoC based Hardware All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum