easyddup a simple upgrade/downgrade utility

Post new topic   Reply to topic    DD-WRT Forum Index -> Contributions Upload
Goto page Previous  1, 2, 3, 4 ... 18, 19, 20  Next
Author Message
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Wed Mar 10, 2021 13:44    Post subject: Reply with quote
yoyoma2 wrote:
Then the progress status doesn't matter as much. I can't do more than tweak a few lines.

I've found another script that can output the progress here https://github.com/axelabs/splitcurl
Also, joining the parts takes some time, do you want to add the status update similar to the screenshot below?

EDIT: actually axel (Entware package) is better than curl in splitting download into multiple parts

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)
Sponsor
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Wed Mar 10, 2021 19:46    Post subject: Reply with quote
New version posted in OP (March 10th).

Changes:

Added your file joining progress messages and the downloading progress messages from the other script you found. It really looks more polished with the progress messages. Thanks for the free code.

Set the default number of parts to 8 in the sample easyddup.ini file. The cool graph shows there's significant gain with 8 and not everyone will have as powerful a router as you.

Notes:

Suggestions are welcome, just don't be disappointed if only small things get done. Yes an animated gif to replace the old image in the OP would be great.

The best part of easyddup is the nvram tool ported from another project (see OP). It even has a migration mode for setting up a new/different router. Manually entering many settings should be obsolete.

Unfortunately the number of tech savvy dd-wrt users is dwindling so people such as yourself who can tinker under the hood are fewer and fewer but some stuck around.
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Thu Mar 11, 2021 14:23    Post subject: Reply with quote
In the GIF I only show the INI file and normal FW upgrade. If you want to show anything else please let me know. I think it's better to append than to replace the old image.
Actually I think that the FW upgrade feature is also very convenient. Previously I always have to:
  1. manually download the new FW file (this step includes digging into the list of devices and clicking on the link to the fW file),
  2. open ddwrt control panel, navigate to Admin > Backup tab, click on Backup, choose where to save the file (in case something goes wrong with the new FW), then
  3. navigate to Upgrade tab, browse to the new FW file and click on Upgrade.

Surely it's not complicated, but repeating all these steps is really boring and unproductive, especially if you want to keep up with BS new builds (at most a few days for a new build). It's 2021 now, AI is everywhere, there's a virtual personal assistant in every mobile phone, cars can drive by themselves, so if this FW upgrade can't be 100% automated then it should at least be effortless and enjoyable.

For cleaning up the nvram backup, I think there should be another script to check & cleanup the duplicated files, then there's a param in the ini file to enable/disable calling that new cleanup script. The way to identify the duplicate is to compare the hashes of nvram-all-* files. If the nvram-all- are the same then the nvram-ini-, nvram-restore-, nvram-usr- should be identical also. Only the older duplicate files should be deleted so that nvram-restore-latest.sh is not broken. Probably it's like this:
  1. sort nvram-all-* files by name in reverse order (ls nvram-all-* | sort -r)
  2. for each nvram-all- file in sorted list
    1. get unique part of file name (date time mac behind the nvram-all- prefix)
    2. get hash (md5sum from busybox)
    3. compare with previous file's hash (null if first file)
      1. if same hash: delete nvram-all-, nvram-restore-, nvram-usr- with the same date time mac above
      2. else previous file's hash = current file's hash

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)


Last edited by IONK on Thu Mar 11, 2021 16:42; edited 1 time in total
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Thu Mar 11, 2021 16:30    Post subject: Reply with quote
Added the demo gif to the OP, thanks! It's perfect but the forum's banner limits the useful vertical pixels so on a 1080p monitor, a tall image is harder to see. You might want to remove your post's copy to save space.

Good suggestion for a cleanup script but not making any promises.

How about an auto_select_latest ini option in upgrade mode. Would need to make sure it's not already installed however. In downgrade mode manual selection is necessary.

Don't assume the backup created by easyddup saves all the settings you care about. The OP warns that the ini file that defines what to save needs contributions/augmenting.
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Thu Mar 11, 2021 17:14    Post subject: Reply with quote
yoyoma2 wrote:
Added the demo gif to the OP, thanks! It's perfect but the forum's banner limits the useful vertical pixels so on a 1080p monitor, a tall image is harder to see. You might want to remove your post's copy to save space.
I restrained the width to fit the 768px rule but I totally forgot about the height. Too bad, lazy mode has kicked in, I'll record another one if there is a new update.
Can I add this to my signature?
Quote:
A simple upgrade/downgrade utility by yoyoma2

yoyoma2 wrote:
How about an auto_select_latest ini option in upgrade mode. Would need to make sure it's not already installed however. In downgrade mode manual selection is necessary.
Definitely I will use it. Just a side note, with that option it will be very tempting to set up a cron job for auto FW upgrade, which is not yet clear if it's salvation or gateway to doom
yoyoma2 wrote:
Don't assume the backup created by easyddup saves all the settings you care about. The OP warns that the ini file that defines what to save needs contributions/augmenting.
sorry I didn't read carefully and I thought the nvram-all- file contained everything. Do you know the underlying reason that not all settings are saved?
_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)


Last edited by IONK on Sat Mar 20, 2021 5:37; edited 1 time in total
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Thu Mar 11, 2021 23:25    Post subject: Reply with quote
New version posted in OP (March 11th).

Changes:

Added EDDUP_AUTO_SELECT_LATEST and renamed EDDUP_AUTO_SELECT to EDDUP_AUTO_SELECT_FILE to avoid confusion so edit your existing easyddup.ini file accordingly.

Theoretically a cron job could now be used to fully automate updates just like kong's ddup permitted. I would try this on a secondary tinkering router and not your main one. Kong builds were more rare and more tested. BS builds are much more frequent and slightly better than developer nightly builds.

Notes:

Yes the "all" files have everything. I meant the other files have only the nvram variables listed in the nvram-dd-wrt.ini file. These variables are safe between build numbers and even between routers in some cases.

Sure that line in your signature would be cool. I edited the demo gif comment to give credit.
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Fri Mar 12, 2021 16:12    Post subject: Reply with quote
I think there is a typo on line 19 in easyddup.sh

EDIT: update GIF
EDIT: deleted GIF. I should have attached the compressed version but somehow I ended up posting the uncompressed version, probably because I was sleepy and there was this ninja thing. Probably people won't mind 60kB extra.

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)


Last edited by IONK on Sat Mar 13, 2021 0:26; edited 3 times in total
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Fri Mar 12, 2021 18:22    Post subject: Reply with quote
@IONK
I put your new demo in the OP. A complete update by typing a single command. Very nice gif! You can delete your post's copy or is that ninja editing?

Commented out line 19 which is used for testing. Embarassed

I like your new signature. Very Happy
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Sat Mar 13, 2021 5:23    Post subject: Reply with quote
Do you know how to get the firmware version in the format below via SSH/telnet?
Code:
DD-WRT v3.0-r45993 std (03/12/21)

I want to create a small script to auto output these info to a text file when I do the FW upgrade, so that I can post to the forum as below:
Quote:
Router/Version: D-Link DIR-868L Rev.A1
File/Kernel: dir868a-webflash.bin / Linux 4.4.260 #2216 SMP Tue Mar 9 12:00:13 +07 2021 armv7l
Previous/Reset: DD-WRT v3.0-r45955 std (03/09/21) / no
Mode/Status: Repeater Bridge on 5GHz AC/N-Mixed (wl0 is purposely turned off) + USB Storage / working
Issues/Errors: none

This is what I have added to easyddup.sh:
Code:
write_report () {
   reportdir="$buildcache/$selection/$board"
   reportfile=report_$buildno.txt

   echo "[b]Router/Version: [/b] $boardraw $revsuffix" > $reportdir/$reportfile
   echo "[b]File/Kernel: [/b] $fselection / kernel_string_uname_srvm" >> $reportdir/$reportfile
   echo "[b]Previous/Reset: [/b] `echo "$builds" | grep "\-r$buildno"` / no" >> $reportdir/$reportfile
   echo "[b]Mode/Status: [/b] Repeater Bridge on 5GHz AC/N-Mixed   (wl0 is purposely turned off) + USB Storage / working" >>  $reportdir/$reportfile
   echo -n "[b]Issues/Errors: [/b] none" >>  $reportdir/$reportfile
}

The output report file I get is:
Code:
[b]Router/Version: [/b] Dlink-DIR868L reva
[b]File/Kernel: [/b] dir868a-webflash.bin / kernel_string_uname_srvm
[b]Previous/Reset: [/b] 03-12-2021-r45993 / no
[b]Mode/Status: [/b] Repeater Bridge on 5GHz AC/N-Mixed   (wl0 is purposely turned off) + USB Storage / working
[b]Issues/Errors: [/b] none

I've also added the command below to the startup script, so that it can output the the kernel version to the report file after reboot:
Code:
find /opt/easyddup/ -type f -name "report_*.txt" | tail -1 | xargs -r sed -i "s/kernel_string_uname_srvm/$(uname -srvm)/g"

The result is close enough for me, but I still prefer to get the correct firmware version string as displayed on ddwrt web control panel and in the "Pre-authentication banner message" when connecting to ddwrt via ssh/telnet, because that full string also has the "std" build variation, not just the build date and build number.

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Sat Mar 13, 2021 13:57    Post subject: Reply with quote
This is a start to get the distribution type and format string for printf:
Code:
nvram get dist_type
strings /usr/sbin/httpd | grep 'DD-WRT ' | grep -v 'Control Panel' | grep -v Development
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Sat Mar 13, 2021 15:11    Post subject: Reply with quote
yoyoma2 wrote:
This is a start to get the distribution type and format string for printf:
Code:
nvram get dist_type
strings /usr/sbin/httpd | grep 'DD-WRT ' | grep -v 'Control Panel' | grep -v Development
Thanks! Now I only need to play around with the send email template included in easyddup. The final goal is that it can automatically check for new version, upgrade FW, and send me the email with the filled in form so I can post to the "New Build" thread.
On second thought, maybe I can also let it login to the forum, find the correct "new build" thread, and post the form automatically.

P/S: how did you find out the "strings /usr/sbin/httpd"? I tried to search in the source code of ddwrt on github but I couldn't get the answer by myself.

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 359

PostPosted: Sat Mar 13, 2021 19:10    Post subject: Reply with quote
When I posted this screenshot I had looked through the dd-wrt code so by chance you asked something I already knew.

I was thinking of adding hook(s) so user code could be included/run such as your reports.

Are the new build forum threads created by anyone who happens to notice a new build for their platform? Is it possible you notice the new build before the thread even exists?
blkt
DD-WRT Guru


Joined: 20 Jan 2019
Posts: 5555

PostPosted: Sat Mar 13, 2021 20:07    Post subject: Reply with quote
yoyoma2 wrote:
Are the new build forum threads created by anyone who happens to notice a new build for their platform?
Only manually watch Trac SVN for added support, then confirm the new build folder and files exist at release.
yoyoma2 wrote:
Is it possible you notice the new build before the thread even exists?
Challenge accepted. =) Automated build thread reports are pointless without user feedback over time and logs.
IONK
DD-WRT Guru


Joined: 19 Aug 2011
Posts: 899

PostPosted: Sun Mar 14, 2021 1:44    Post subject: Reply with quote
blkt wrote:
Automated build thread reports are pointless without user feedback over time and logs.
I think most people don't bother to report after upgrading FW if they don't encounter any issue. Also, I think even my manual reports' credibility is questionable, because I only use a very small subset of features available on DD-WRT, with specific settings that are not reflected in the reports. Should I wait for 1 hour or 1 day or 1 week to confirm the stability? Should I add more devices to my network and do some stress test to check the throughput? Should I run some automated vulnerability scanning tool to verify the basic security?
For me, as long as it can boot up after FW upgrade and my 2 devices are connected and I can access the internet then it's considered a milestone.

EDIT: probably I can also let the script scan through the kernel log/syslog and attach the logs to the report if there's any error not seen before.

_________________
▫ RSS feed for DD-WRT releases (2023): https://politepol.com/fd/bdQnjdZgyaln
easyddup - A simple upgrade/downgrade utility by yoyoma2 --- as featured in 📌 List of reference threads, how-tos, and otherwise (Moderator's Pick 🌟)
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 13904
Location: Texas, USA

PostPosted: Sun Mar 14, 2021 2:19    Post subject: Reply with quote
blkt wrote:
Automated build thread reports are pointless without user feedback over time and logs.

Agreed.
IONK wrote:
I think most people don't bother to report after upgrading FW if they don't encounter any issue.

People don't report after flashing and then we have people asking, "What's the latest build that works / is stable for my router?"... flash and report. Not doing so is a complete disservice to the forum and community at large and it is cumbersome dealing with the lack of reporting (because everyone was horning in on the device threads and creating a cesspool of (mis)information). I personally have gotten in the habit of watching release threads and development and timing my upgrades around once a month. Only recently have I upgraded more often due to specific issues.

_________________
"Life is but a fleeting moment, a vapor that vanishes quickly; All is vanity"
Contribute To DD-WRT
Pogo - A minimal level of ability is expected and needed...
DD-WRT Releases 2023 (PolitePol)
DD-WRT Releases 2023 (RSS Everything)

----------------------
Linux User #377467 counter.li.org / linuxcounter.net
Goto page Previous  1, 2, 3, 4 ... 18, 19, 20  Next Display posts from previous:    Page 3 of 20
Post new topic   Reply to topic    DD-WRT Forum Index -> Contributions Upload All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum