Joined: 05 Oct 2008 Posts: 646 Location: Helsinki, Finland / nr. Alkmaar, Netherlands
Posted: Thu Aug 31, 2023 11:22 Post subject: why shutdown commands get executed twice in some cases?
On my R7800 running the latest build r53445 I have a few shutdown commands which copy a script from /jffs to /tmp/root and then execute that script. Said script changes the working directory to /, stops mdns and samba3 services, unmounts /jffs and /opt, performs e2fsck checks on the corresponding devices and then ends so the router can restart.
It also writes some messages to the log for me to see how the router is proceeding.
I have the router send the syslog to a server on my LAN, which is how I can see what happens on the router.
In case a reboot is initiated from an SSH session or as a scheduled reboot (GUI), the shutdown commands are executed twice in a row before the router reboots.
If, however, I reboot the router from the button in the Administration page of the GUI, the shutdown commands are executed only once before the router restarts (as one would expect in the first place).
(There's yet another way to reboot the router, viz. by a 'reboot now' command in the command panel in the GUI. I tried that, too, but I forgot if the shutdown commands were executed once or twice in that case.)
ADD: running a reboot command from the GUI causes the shutdown commands to execute just once, just like in case of the reboot button in the GUI.
Joined: 05 Oct 2008 Posts: 646 Location: Helsinki, Finland / nr. Alkmaar, Netherlands
Posted: Fri Sep 01, 2023 10:35 Post subject:
The shutdown script I run from the shutdown commands in the GUI contains stuff related to the jffs (and optware (entware) partition on the thumbdrive in my R7800.
Running those commands a second time is not needed and, who knows, may have side effects.
As the current build insists on running the shutdown commands twice for some of the various ways to reboot the router, I now use the following commands in the shutdown panel of the GUI:
if mount | grep -qw "/jffs"; then
logger -t --- "Copy myShutdown.sh from /jffs to /tmp/root ..."
cp /jffs/command_scripts/myShutdown.sh /tmp/root
/tmp/root/myShutdown.sh
else
logger -t --- "No need to execute myShutdown.sh twice in a row ..."
fi
(indentation lost in posting)
There are other ways to find out if the jffs partition is mounted (if clause), but some, like mountpoint and findmnt, require installing from entware. All fine and well, but my shutdown script also unmounts (and checks) the /opt partition (or, rather /dev/sda2), so findmnt or mountpoint would be gone the second time that dd-wrt runs the shutdown commands in the GUI.
Below is my shutdown script, which is copied to RAM (/tmp/root) right before execution. This way I can edit it and the latest version will be run from /tmp/root upon rebooting.
#!/bin/sh
# AR - 02-05-2022 ... 09-03-2023, 19-07-2023 ... 01-09-2023
# Shutdown script for DD-WRT firmware
# Output from service & e2fsck commands goes to terminal, if any, otherwise it is lost.
# Log exit codes instead.
# Make sure working directory isn't on a thumbdrive partition before unmounting
cd /
# Deactivating Entware (~Optware) ...
# Stop all init scripts in /opt/etc/init.d including symlinks
logger -t --- "$0: Deactivate Entware"
/opt/etc/init.d/rc.unslung stop
logger -t --- "Stop mDNS & SAMBA service before unmounting USB thumbdrive & filesystem checks!"
service mdns stop
service samba3 stop
logger -t --- "$0: Unmount thumbdrive partitions jffs and optware"
umount /jffs
logger -t --- "$0: Exit code for /jffs is $?"
umount /opt
logger -t --- "$0: Exit code for /opt is $?"
logger -t --- "$0: Perform e2fsck -p on sda1 and sda2"
# 'Preen' partitions
e2fsck -p /dev/sda1
logger -t --- "$0: Exit code for sda1 is $?"
e2fsck -p /dev/sda2
logger -t --- "$0: Exit code for sda2 is $?"
logger -t --- "$0: Done checking thumbdrive partitions"
logger -t --- "End of $0 ..."
The last two exit codes should be 0 denoting the thumbdrive fs's are OK (not 'dirty'). This is actually the purpose of this whole exercise, cleaning the file systems on the thumbdrive after a power-off leaves them 'dirty'. It will do its job before a reboot following the power-off, since I find it too tricky to arrange for e2fsck during startup. At least this way the thumbdrive can stay in the router if there is a problem that requires powering off the router (anybody can use the power button on my behalf).
However, if you do not arrange for an rsyslog server to collect syslog output from the router, I expect you won't ever see the output of the shutdown script.
The output issued by the service and e2fsck commands can be seen in an SSH terminal, in case you issue the reboot command from there. It would be nice to be able to direct it to the syslog, but I haven't succeeded in doing that.
I'm open to all comments, except those that say I shouldn't use so much text ending up in NVRAM. I feel the R7800 and XR500 have enough of that. As a human, I need clarity
This reminds me of a ticket I logged a couple years ago about booting up happening multiple times. It was closed as "invalid" saying "write better code". My guess is you will be told to make your script robust enough to handle shutting down multiple times.
Joined: 05 Oct 2008 Posts: 646 Location: Helsinki, Finland / nr. Alkmaar, Netherlands
Posted: Mon Sep 04, 2023 15:07 Post subject:
Well, that's what I did. Make it robust. See the first snippet of code (=shutdown panel contents) in my post.
After the actual shutdown script, the second snippet aka myShutdown.sh, has executed once, the jffs (and the optware) partition is unmounted, so a second time around this shutdown script, containing the umount and e2fsck, will not run.
Not all ways of rebooting the router will make the GUI shutdown commands run twice.
It is all logged to and visible in the syslog, since my rsyslog server (a multifunctional Pi4 with PiJuice UPS) saves the syslog output from the router during all of the reboot process.
If there would be an easy and sensible way to run e2fsck on the thumbdrive partitions at router startup, I would do that. Pulling the power plug causes one or both partitions to have the 'dirty' flag.
In my startup commands I use the 'is-mounted' script (by EGC?) to wait for jffs and optware to be mounted before continuing. However, I don't know of a way to postpone mounting jffs and optware until e2fsck has run succesfully.
I'd migrate the check to the startup commands if I did.
I'd also like to find a way to make the output from the e2fsck command itself go to the syslog instead of to the terminal (if any).
Ideally in the "USB Support" section where "Automatic Drive Mount" is there would be a new "Pre-mount filesystem check" radio button.
I've seen pages of troubleshooting on these forums that would never have happened if corrupt filesystems were not mounted or even repaired when possible.
Joined: 05 Oct 2008 Posts: 646 Location: Helsinki, Finland / nr. Alkmaar, Netherlands
Posted: Mon Sep 04, 2023 19:33 Post subject:
I don't pull the power plug just for finding out what problems it may cause, but it happens that I have to do it. Usually it doesn't cause immediate problems for the subsequent reboot.
When at one time I observed a problem with a thumbdrive partition seemingly being full, I found the FS was 'dirty'. I didn't try to delve into it any further.
Linux disk utility (gmome-disks or something) will repair it, although until not so long ago, it would say it cannot repair the problem, but when I tested it after the repair attempt, the FS was ok again.
I have put more time into trying to get both partitions to show up in dd-wrt as ext2.
There are many tools to see the type of partition but the results differ and then I give up on that subject again until the next time.
I have seen one thumbdrive in my use that dd-wrt reported to have ext2 for both partitions .
Maybe once you do certain things to a thumbdrive it is impossible to make it again like it was when it came from the shop. I've used both MBR and gpt partition tables on my thumbdrives. They're a bit of a mistery part with a mind of their own it seems.
I'd buy new ones just to test and learn, but there are no small 2GB or smaller thumbdrives anymore. Small would equal cheap ... For learning and testing.
Maybe from AliExpress you can get small ones, but I found the thumb drives I got from there to be unreliable, which is the last thing you want.