Posted: Fri Aug 26, 2022 1:12 Post subject: Removing unnecessary/wasterful ^M's from nvram strings
It seems that at least on Windows, multi-line nvram string variables such as keys, certificates, and some multi-line commands have a carriage return (^M) appended to the end of each line harking back to the old DOS '/n/r' convention (though it doesn't seem to be consistent).
Adding somewhat random ^M's to the end of each line is "messy", inconsistent and wastes valuable NVRAM space.
Any suggestions on how to remove them? (beyond writing a kludgy script to rewrite the variables).
Any reason that the default dd-wrt GUI doesn't apply a 'tr' or 'sed' to the inputs to remove these old DOS convention line endings.
What exactly are you using to view / create a text-only version of your nvram variables? This looks like a Windows / Notepad (not Notepad++) problem. I've never noticed this in PuTTY or Terminal. _________________ "The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep." - Robert Frost
"I am one of the noticeable ones - notice me" - Dale Frances McKenzie Bozzio
Windows & Linux handle carriage return, line feed differently. Play with Notepad++, do some research CR LF BOM.
You can guess which OS has more unnecessary and wasteful text files. :) PuTTY Telnet / SSH as root, nvram show.
Enable logs printable output, or click upper left copy all to clipboard, or right click top bar not terminal it will paste.
I actually do all my editing in text-based emacs in Linux (via PuTTy) but I run my browser on Windows. I never use Notepad.
This is coming from cutting & pasting from a PuTTy window into chrome on my browser.
Into "what" in your Chrome Browser? What are you pasting to? I've never seen this copying from PuTTY in Windows and pasting into Notepad++, saving the file and then opening it up in nano or vi. _________________ "The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep." - Robert Frost
"I am one of the noticeable ones - notice me" - Dale Frances McKenzie Bozzio
Actually, I can reproduce this without any cutting or pasting...
I tried typing a random multiline string *directly* into a textbox in the GUI from a chrome browser window on a Win11 PC. For example, I typed:
Code:
hello
goodbye
*directly* into "Additional Configuration" under Services/VPN without any editor.
Then running 'ssh <router> nvram get openvpn_config | cat -v' gives:
^M is generated when Windows native apps see LF only (Linux). Problem? Windows uses CRLF. Please do research.
I know that. I have been using Linux for almost 30 years, since version 0.99. I don't need to "do research" to know about dos vs. linux conventions for end of line..
As per my earlier post, dd-wrt is being inconsistent.
It seems like the \n\r is being introduced by chrome under Windows but sometimes dd-wrt filters out the \r (e.g., in the 'commands' textbox) and sometimes it doesn't.
Typing the same strings into 2 different text boxes shouldn't have such different behaviors.
Into "what" in your Chrome Browser? What are you pasting to? I've never seen this copying from PuTTY in Windows and pasting into Notepad++, saving the file and then opening it up in nano or vi.
Replicate what I did -- no cutting or pasting.
Simply type a multi-line string into the text boxes I mentioned in Chrome under Windows.
<facepalm> Then do not use Windows Terminal SSH. Do not use Windows Chrome running commands in WebUI.
<facepalm> Nothing to do with SSH.
It happens so long as you use *ENTER* text (whether directly typing or pasting) into the GUI when running on a Windows browser.
I confirmed this behavior in both Chrome and Edge.
So basically, unless the answer is not to use Windows, then I maintain that this is a bug or at least undesirable behavior and an inconsistency in how the dd-wrt GUI reads in text strings.
Any half-decent program will filter out the useless \r characters.
It seems that dd-wrt does this for some GUI text fields but not others.
I posted a way to manually fix this after the fact so it's no longer an issue for me.
The right fix would be to chomp the final character in any line that ends in \r.
Best practice is to do that so that strings are formatted and stored consistently independent of which browser/OS is used to enter them.
If the moderators/DD-WRT Gurus think the current behavior is the desired behavior then so be it... you can bring a horse to water...
I've also wondered why dd-wrt nvram variables use CR LF line endings for multi-line variables instead of the native linux standard. Even though my "Additional Jobs" are entered from firefox linux, the following shows a ^M after each line. A hex editor shows the familiar DOS 0D 0A line endings.
Code:
root@DD-WRT:~# nvram get cron_jobs > var.bin
root@DD-WRT:~# vi var.bin
Perhaps the goal was to make nvram dumps more readable for Windows users? Even WordPad from decades ago could display linux text files properly so not much use.
Someone could ask BS or log a bug (it's not really a bug).
PS. There's a function in the code (forgot where) that translates any type of line endings in nvram variables to \n before putting them in the GUI so your hack should work until dd-wrt puts the CRLFs back in.
I've also wondered why dd-wrt nvram variables use CR LF line endings for multi-line variables instead of the native linux standard. Even though my "Additional Jobs" are entered from firefox linux, the following shows a ^M after each line. A hex editor shows the familiar DOS 0D 0A line endings.
Code:
root@DD-WRT:~# nvram get cron_jobs > var.bin
root@DD-WRT:~# vi var.bin
Perhaps the goal was to make nvram dumps more readable for Windows users? Even WordPad from decades ago could display linux text files properly so not much use.
Someone could ask BS or log a bug (it's not really a bug).
PS. There's a function in the code (forgot where) that translates any type of line endings in nvram variables to \n before putting them in the GUI so your hack should work until dd-wrt puts the CRLFs back in.
And my hack works until/unless you re-save/re-apply the settings on a page that has such variables.
I agree it's not really a bug in that nothing is broken - it's just messy, inconsistent, and a waste of a char per line when nvram can be scarce.
I seems ridiculous that cron_jobs adds a \r while rc_startup doesn't -- what rationale is there for that????
The fix_cr function called when getting multi-line nvram variables is here. I totally agree it should be called when setting nvram variables to save space and be consistent with the native platform.
It's probably a browser thing that dd-wrt chose to fix on the way out of nvram rather than on the way into nvram. Log a bug pointing out the nvram savings aspect and that your hack doesn't cause problems. Worst-case it gets closed as not a bug.
