Thanks... fixed! Does anyone know if there is bash `strict` mode (a la "use strict" in JavaScript files)? Because , honestly, my router does not report that as an error.
Thanks
Maybe is the busy box version that makes the difference (mine is BusyBox v1.22.1 on K3.x DD-WRT) and I believe is the same as the previous user that mentioned the problem (R7000 runs k3.x latest kernel/DD-WRT)
After seeing the error (telnet/ssh using terminal/macos) I set debug mode on for the yamon script to locate the line with the problem by running
Joined: 05 Mar 2014 Posts: 193 Location: Edmonton, AB
Posted: Tue Apr 08, 2014 16:20 Post subject:
Yes, my Busybox version is 1.22.1.
@Al: Do you want me to edit that detanod to _detanod in my script as @spirosbo suggested? Also, another quick question. I did download your 2.04d but web report page says version 2.03. I thought it is just the report part of YAMon2 rather than the script itself.
Mike et al: The `Error #4 reading...` messages get displayed when the JavaScript cannot find a specified data file. I've found that it is virtutally inevitable that the JavaScript in the browser will try to read the file while the script on the router is updating it and this causes errors. Rather than displaying the unhelpful browser errors, I wait for a second or two and try to read the file again... after 3-4 consecutive unsuccessful attempts, I display the `Error #4 reading...` message
I'll give your steps a try later and report back. Would it be possible to update the message so it reports the entire path/file name? Right now the error only says "data/users.js". If I had the whole path, I'd know where to look as those files are in multiple places.
OK, after starting it from PuTTY, it started working. There are some oddities though:
(1) Top of the yamon2.html says it is version 2.0.3. Maybe you just forgot to change it?
(2) I get two red stickers on the yamon2.html page saying "a is null" or maybe "s is null". Hard to tell because the text is so tiny.
(3) On the "Daily Usage" tab, sometimes when I enter the page it shows "new device" and the Mac address for every entry. Other times it shows "New device" only. Timing issue for when the file is being read?
I'm particularly interested in why it says version 2.0.3 though. Where is that coming from? It's not in the download!
Joined: 05 Mar 2014 Posts: 193 Location: Edmonton, AB
Posted: Wed Apr 09, 2014 17:40 Post subject:
I have a small update on this that I am writing here so that it may help someone else who might consider installing this script. I had to update the DD-WRT version because of the VPN security issue so I thought I take Al's suggestion on formatting the USB flash drive in ext2 format. With the default(fat32) format I had the USB drive load as /tmp/mnt/sda and /tmp/mnt/sda1 but could not get it to load to /opt.
Even when I navigated to /opt I could not create YAMon2 directory there due to permission issues. Anyway, I completely wiped the same USB drive(8GB) and formatted it using gparted into ext2 partition(single partition for now) and after updating the router to Kong's latest build and enabling the usb etc, I inserted the newly ext2 formatted flash drive.
Within seconds, it was mounted to /opt/sda and /opt/sda1. I quickly checked if I had write privilege and I sure did. Then I proceeded to install YAMon2 all over(from a freshly downloaded zip file hoping to see if there was any change in the 2.03 vs 2.04 in the report). The installation was much simpler compared to my earlier struggles because of two reasons. First, the /opt/YAMon2 directory is the default in Al's scripts so I didn't have to make any changes. Second, I knew a little more this time around. Overall, I can say that it is a lot easier to have your USB mounted on to /opt with the ext2 formatted drive.
I still got that bad number error but as soon as I changed those two values that Al had suggested( _log2file=2 and _loglevel=0 in config.file) everything started working.
I spent hours the first time and only minutes this time.
Mike et al: The `Error #4 reading...` messages get displayed when the JavaScript cannot find a specified data file. I've found that it is virtutally inevitable that the JavaScript in the browser will try to read the file while the script on the router is updating it and this causes errors. Rather than displaying the unhelpful browser errors, I wait for a second or two and try to read the file again... after 3-4 consecutive unsuccessful attempts, I display the `Error #4 reading...` message
I'll give your steps a try later and report back. Would it be possible to update the message so it reports the entire path/file name? Right now the error only says "data/users.js". If I had the whole path, I'd know where to look as those files are in multiple places.
Mike
Hey Mike,
Not sure but I think that this path is relative to the main dir. Also, I left you a PM, not sure if you received it...
Joined: 05 Mar 2014 Posts: 193 Location: Edmonton, AB
Posted: Thu Apr 10, 2014 19:14 Post subject:
Hi Al, I know you are busy so no hurry in answering this. I just wanted to know if having vpn run on the router for one computer would throw off logging the download in someway. Yesterday I noticed that YAMon2 was working fine in all respects except it never updated the download in one of the computers at all. I downloaded a few gigabytes yet there was no change in that computer but all the others were regularly updated even if was a few KB.
Hi Al, I know you are busy so no hurry in answering this. I just wanted to know if having vpn run on the router for one computer would throw off logging the download in someway. Yesterday I noticed that YAMon2 was working fine in all respects except it never updated the download in one of the computers at all. I downloaded a few gigabytes yet there was no change in that computer but all the others were regularly updated even if was a few KB.
I can confirm YAMon2 not registering/logging VPN/PPTP traffic. PPTP connections don't show as connected devices
Al_c, what are you using to measure traffic? YAMon seems to be off about 5-10% of total traffic for the day when compared to dd-wrt baked in traffic metrics. Im assuming that it doesnt log and traffic direct from router, right?
Joined: 05 Mar 2014 Posts: 193 Location: Edmonton, AB
Posted: Thu Apr 10, 2014 21:43 Post subject:
Hi, Thanks for confirming this. I wasn't sure if it was VPN or some setting in my system. Funny thing, in my case, is that I was checking it and at one session it logged about a GB and I closed the Yamon session, put the computer to sleep and in the evening it wouldn't record anything even though there was download and the netgear R7000 was showing activity in the 10-15mbps.
Hi Al, I know you are busy so no hurry in answering this. I just wanted to know if having vpn run on the router for one computer would throw off logging the download in someway. Yesterday I noticed that YAMon2 was working fine in all respects except it never updated the download in one of the computers at all. I downloaded a few gigabytes yet there was no change in that computer but all the others were regularly updated even if was a few KB.
I can confirm YAMon2 not registering/logging VPN/PPTP traffic. PPTP connections don't show as connected devices
Al_c, what are you using to measure traffic? YAMon seems to be off about 5-10% of total traffic for the day when compared to dd-wrt baked in traffic metrics. Im assuming that it doesnt log and traffic direct from router, right?
YAMon uses an 'iptables' rule to capture the data.
In brief, the way that iptables works is that rules are added to various chains. The first rule that matches incoming/outgoing traffic processes/routes the packets. The YAMon rule is added to the FORWARD Chain.
So, if you have rules on your router that route data before the FORWARD chain, then YAMon never sees it.
Do you have UPnP enabled in the DD-WRT GUI? See the NAT/QoS-->UPnP tab. UPnP allows apps to create their own rules (behind your back). When I disabled this option, I found that data on my router was a closer match to the Traff daemon values.
I will look into moving the YAMon rule into an earlier chain but I have no idea ATM how easy/difficult this could be, how hard it would be to manage within the script or if it would impact traffic on your routers. (The all of the precursors to YAMon did not do this so I'm assuming it is not that easy)
Joined: 05 Mar 2014 Posts: 193 Location: Edmonton, AB
Posted: Fri Apr 11, 2014 15:52 Post subject:
Thanks, Al. I am new to this so I don't know much. I checked and upnp is not enabled in my router. What may help you is the fact that Yamon version 1(your latest) absolutely ignores all vpn traffic while version 2 sometimes catches it. I had two sessions on a pc of which one was recorded properly and the other not.
The way I have it is as follows:
I have a vpn client running in the router. Then, in that page there is a column for policy based routing. I have a pc and a bluray player's ip inserted there. So, those two go through the vpn and the rest go through the isp gateway.
I am sure those are converted into iptables but I don't know enough.
Finally, I want to make it clear that I wasn't complaining but just reporting so you know how it functions. I am using it strictly for fun and educational purposes and I appreciate what it does already. I don't depend on it for any real ISP billing needs.
I asked this a while back but maybe it got missed. Is there any way to change one of the iptables commands so that all traffic (LAN and WAN) is captured and not just WAN traffic? Most people have unlimited WAN bandwidth and in that case (my case) I'd find it much more useful to see what LAN traffic is eating up local bandwidth.
Also still interested in the answer to my question: why does the yamon2.html report say 2.0.3 when I installed 2.0.4d. It appears to use yamon_version id in the html but that id is not defined anywhere in the downloaded materials. Is it pulling that from an offsite location?
I asked this a while back but maybe it got missed. Is there any way to change one of the iptables commands so that all traffic (LAN and WAN) is captured and not just WAN traffic? Most people have unlimited WAN bandwidth and in that case (my case) I'd find it much more useful to see what LAN traffic is eating up local bandwidth.
Also still interested in the answer to my question: why does the yamon2.html report say 2.0.3 when I installed 2.0.4d. It appears to use yamon_version id in the html but that id is not defined anywhere in the downloaded materials. Is it pulling that from an offsite location?
Thanks,
Mike
Mike - I've recently changed things so that I have separate version variables for the script and the reports. The reports already get their version number from yamon.js. When I get around to posting 2.0.5, the script will write its version to config.js and that will get displayed in the reports as well. In general, I will try to keep them in sync (but that won't always be the case). Hopefully this will not be too confusing.
WRT to the WAN & LAN traffic, I will have to look into that further. As explained in a previous message, the way that yamon.sh works is that it records all traffic that it sees in the FORWARD chain - see `iptables -L FORWARD` to see all of the rules in the FORWARD chain
see `iptables -L` for all chains on your router.
Google `iptables` for more info than you'll ever want/need about the topic.
WRT to the WAN & LAN traffic, I will have to look into that further. As explained in a previous message, the way that yamon.sh works is that it records all traffic that it sees in the FORWARD chain - see `iptables -L FORWARD` to see all of the rules in the FORWARD chain
see `iptables -L` for all chains on your router.
Google `iptables` for more info than you'll ever want/need about the topic.
Looks like the FORWARD chain specifically says "non local". I tried setting --table raw and using PREROUTING. Some improvement but still not right.
BTW, when you update the usage, do you look in the ARP table in /proc/net/arp and only traverse the connected devices in that list and get the updates only for those? If so, that might be why some of the numbers are off. I'm finding /proc/net/arp to be very unreliable for discerning which devices are currently connected. Also, a device could connect and transmit hundreds of megabytes of data, and then disconnect before your update routine. When it disconnects, it disappears from the ARP list after only a few seconds of being inactive.
So... a device could download or upload a lot of data, disconnect, and the next time you do your update, you check the ARP table and that device is not listed and hence not updated and you miss that chunk of data if it disconnects before you update. I'm not a shell script pro but it looks like that's how you are doing it. If so, it should probably traverse the MACs and IPs in the users file instead of the ARP table. That way at least if it is a device that has been "noticed" before, it won't get missed in the updates because it's not in the ARP cache by the time the update gets around to reading the ARP table.
