Corrupted files after a power failure

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> General Questions
Author Message
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2038
Location: Ottawa Canada

PostPosted: Mon Apr 20, 2015 4:09    Post subject: Corrupted files after a power failure Reply with quote
We had a brief power failure last night and when things came back online I found that a couple of data files for my YAMon script were left in a bad state.

The files were on an 8GB USB drive (formatted ext2).

`fsck` identified the corrupted files but the messages from that utility say that running it against a mounted drive will cause more damage and I could not figure out how to properly `umount` the drive...

Eventually I opted for a brute force solution - i.e.,
- pulled the USB drive out of the back of the modem,
- plugged it into my laptop,
- backed things up
- wiped the partition
- reformatted the drive
- restored the data
- plugged the drive back into the router
- rebooted the router

That got me up and running again but I figure that there has to be a better way to do this.

Any suggestions?

TIA

Al
Sponsor
KrypteX
DD-WRT Guru


Joined: 18 Jan 2014
Posts: 2411

PostPosted: Mon Apr 20, 2015 5:47    Post subject: Reply with quote
I saw the same thing a couple of times, on power outage. The only difference: the USB stick was formatted FAT32. Had to unplug it forcefully (couldn't unmount first with the USB unmount button), plug it into laptop, run Check Disk, things got fixed, unmount properly from laptop, connect it back into router, things look fine. I didn't have to format it.
_________________
THERE ARE NO STRANGERS HERE; ONLY FRIENDS YOU HAVEN'T YET MET.
________________________________________________________________________________________________________
DD-WRT CHANGELOG | DEVICES | DD-WRT BUILDS | KONG BUILDS | UNOFFICIAL BUILDS | DD-WRT in VIRTUALBOX

Testing BrainSlayer's DD-WRT build 31924 on various units:
NETGEAR : WNDR3800 WNDR3800CH WNDRMACv2 WNDRMACv1 WNR2200 WNR2000v3
TP-LINK : ArcherC7v3 WR1043NDv3 WR740Nv4 WR740Nv6 WR710Nv1 WR810Nv1 WA850REv1 WA901NDv3 WA901NDv4
D-LINK : DIR-862L DIR-615D2
ASUS : RT-N10 C1
VirtualBox : x86_64
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2038
Location: Ottawa Canada

PostPosted: Mon Jun 01, 2015 3:54    Post subject: OK... it happened again Reply with quote
We had another power failure today and once again two of my YAMon data files were corrupted on my router.

The corrupted files were not visible in the directory and would report an input/output error if I tried to access or delete them. I struggled for some time to figure out how to umount the USB drive so that I could repair the corrupted files.

It was much easier this time...
1. I opened the DD-WRT GUI and changed to the Services-->USB tab
2. I disabled the `Automatic Drive Mount` radio button and then saved & applied the settings (and just to be safe, rebooted the router)
3. When the router was back online, I opened a PuTTY window and performed a `fsck` on my USB drive - in particular, I entered `fsck -y /dev/sda5` (the -y option automatically selects yes when prompted to fix an issue)
4. to be safe, I then ran that command again.
5. I returned to the DD-WRT GUI-->Services-->USB and enabled the `Automatic Drive Mount` radio button; saved & applied the settings (and, rebooted the router)

Fortunately, this found and fixed a whole mess of issues (check the Lost & Found directory on your USB drive because there might be a mess of recovered files in that directory).

Hopefully this helps if you encounter a similar issue.

Al


Last edited by al_c on Sat Apr 16, 2016 15:14; edited 2 times in total
tsaylor
DD-WRT Novice


Joined: 20 Jan 2014
Posts: 19

PostPosted: Sun Jun 07, 2015 18:49    Post subject: Reply with quote
Thanks Al. I've had this scenario a few times as well, and once recently, when I discovered my YAMon had been offline for a few weeks due to a missing users.js. My router doesn't have fsck (I suppose it could be added), so I've used an Ubuntu flash drive to access YAMon's drive on a PC. Unfortunately most of my PC's won't boot that Ubuntu disk but that's another story.

Anyway, fsck got things running again but my users.js changed from corrupted to entirely missing. Apparently fsck couldn't salvage it. Fortunately I found a prior copy under the www folder and was able to recover by copying that file back to the standard location.

My assumption is that these files get corrupted during power loss because they are in the midst of being written to. I'm not sure what you can really do about that, other than perhaps limiting the frequency of writes to the filesystem.

The other thing is the filesystem type. I'd chosen ext2 because it is supposed to make the drive last longer due to less frequent writes. But now I'm reading that this is much less stable and likely to corrupt during power loss. A journaled filesystem is recommended for that reason. Maybe ext4? It seems like setting a different recommended filesystem could be key here, and probably the closest thing to a real solution.
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2038
Location: Ottawa Canada

PostPosted: Tue Jun 09, 2015 15:24    Post subject: Reply with quote
tsaylor wrote:


The other thing is the filesystem type. I'd chosen ext2 because it is supposed to make the drive last longer due to less frequent writes. But now I'm reading that this is much less stable and likely to corrupt during power loss. A journaled filesystem is recommended for that reason. Maybe ext4? It seems like setting a different recommended filesystem could be key here, and probably the closest thing to a real solution.


I'd shied away from ext3 for the same reasons that you've mentioned above.

But, my router was rebooted unexpected again yesterday and when it came back up, my user.js and hourly date files were corrupted.

I figured I'd bite the bullet and change the FS. I backed everything up and reformatted my 8GB USB drive to ext4 (using MiniTool Partition Wizard - the free home use version http://goo.gl/xhGaX0) .

Apparently you can turn-off the journalling in ext4 although I was not able to find a switch for that in the MiniTool UI so AFAIK, I have journalling.

Anyone know how to turn it off?

I've reattached the drive to my router and I'll let you know how it goes.

Al
KrypteX
DD-WRT Guru


Joined: 18 Jan 2014
Posts: 2411

PostPosted: Tue Jun 09, 2015 15:37    Post subject: Reply with quote
Just a quick question: Why aren't you using FAT32 or NTFS ? I know FAT32 limits the max file size to 4GB, but still...
_________________
THERE ARE NO STRANGERS HERE; ONLY FRIENDS YOU HAVEN'T YET MET.
________________________________________________________________________________________________________
DD-WRT CHANGELOG | DEVICES | DD-WRT BUILDS | KONG BUILDS | UNOFFICIAL BUILDS | DD-WRT in VIRTUALBOX

Testing BrainSlayer's DD-WRT build 31924 on various units:
NETGEAR : WNDR3800 WNDR3800CH WNDRMACv2 WNDRMACv1 WNR2200 WNR2000v3
TP-LINK : ArcherC7v3 WR1043NDv3 WR740Nv4 WR740Nv6 WR710Nv1 WR810Nv1 WA850REv1 WA901NDv3 WA901NDv4
D-LINK : DIR-862L DIR-615D2
ASUS : RT-N10 C1
VirtualBox : x86_64
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2038
Location: Ottawa Canada

PostPosted: Tue Jun 09, 2015 17:04    Post subject: Reply with quote
KrypteX wrote:
Just a quick question: Why aren't you using FAT32 or NTFS ? I know FAT32 limits the max file size to 4GB, but still...


Why? Because the DD-WRT documentation I read said it should be ext2. So, being a DD-WRT newbie, that's what I did...

If ext4 doesn't work out, I'll give FAT32 a whirl.

Al
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2038
Location: Ottawa Canada

PostPosted: Sat Apr 16, 2016 15:13    Post subject: Reply with quote
KrypteX wrote:
Just a quick question: Why aren't you using FAT32 or NTFS ? I know FAT32 limits the max file size to 4GB, but still...


I haven't had a chance to check this out but apparently Fat32 does not support symlinks... to minimize file movements, etc. YAMon creates a number of symlinks so if the initial statement is correct, you cannot use a Fat32 drive for YAMon.

Al
Display posts from previous:    Page 1 of 1
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum