[Howt]

About a week back, I found alexatkinuk's post detailing his ip_conntrack with QoS and bandwidth use reporting.
http://www.dd-wrt.com/phpBB2/viewtopic.php?t=84442&postdays=0&postorder=asc&start=0

I love the thing. So much so that I decided to enhance it. My thanks to the original author.

What's different?

This enhancement - QOS Traffic Monitor - provides for persistent periodic bandwidth reporting by day, host, IP address, or combination thereof. It does not display anything within DD-WRT. Rather, it provides JavaScript data for use in HTML reports.

Daily reporting in addition to point in time.
Less write wear on flash memory - 25 writes/day v. 86,400 (defaults).
Less CPU intensive (simply runs less often).
No data loss upon GUI/Telnet/SSH reboot.
HTML reports with Javascript library for further development.
Hosts defined by DHCP static leases and/or external file.

Everything in the original work by alexatkinuk is retained including the slick MyPage display incorporated into the DD-WRT menus. On the downside, the MyPage display is less responsive. The objective of QOS-TM is geared more towards reporting.

How to install?

A USB drive, JFFS2, or SD/MMC memory is required. A script performs the actual installation. Then, using the DD-WRT GUI, a Startup script, Shutdown script, and Cron Job are defined.

The read-me text file provides the details and can be downloaded here:
http://dl.dropbox.com/u/6624068/QOS_TM_READ_ME.txt

If you would like to examine the setup package (not required as it can be downloaded from within a Telnet/SSH session) is available here:
http://dl.dropbox.com/u/6624068/traff_setup.tar.gz

For accurate identification of hosts, it is strongly suggested that Static Leases and Use JFFS2 for client lease DB be employed. Both are in the DHCP Server section of the Services | Services page. Hosts can also be defined externally in the file traffic_extra_hosts.js.

Confused?

A picture is worth 1,000 words. Here is a screenshot from a demo HTML page. It provides a glimpse of the data arrays made available by QOS-TM. Three "canned" reports are also included and shown in the next post.



In a nutshell, arrays of Javascript data are generated which include:

[Howt]

Good to go!

The issues mentioned previously in this post have been corrected and everything is good to go as detailed in the original post above.

These are the three reports included with QOS-TM. For info on the report options and tools for creating your own reports, see REPORTING.txt in the setup distribution file.

Summary report -
By MAC by Day -
By IP by Day -

My apologies for any inconvenience.

[alexatkinuk]

Thanks, but I suspect the bits of my scripts you are using are the same bits I "borrowed" from wrtbwmonitor. So I'm not sure if I deserve any credit.

Also I just wanted to leave a word of warning. I have found that using iptables to do bandwidth reporting on a per host basis eats a LOT of CPU power as traffic increases.

While I have not been using my DD-WRT script for a while due to switching to OpenWRT, I still use the same iptables rules to capture the data on there.

I have found that on my Buffalo router it would completely max out SIRQs when doing over 40Mbit of traffic, with or without QoS turned on.

So now I am switching to an Atom instead and will be doing something similar to the script on here, except I will just periodically offload the current traffic stats to my NAS over MySQL.

So, if you have problems with the router lagging, DNS failing, then it might be that your router can't handle the traffic monitoring. I was struggling with it for a week or so before it occurred to me to check removing the iptables rules for traffic monitoring.

[Howt]

Thanks for the heads up.

My default installation has traffic_monitor.sh set to sleep 10 seconds between executions versus the original 1 second. For my purposes, it could go as high as 60 seconds since I am more interested in the reports than real-time feedback via the MyPage display... however cool it is ... and it is.

That delay is stored in an NVRAM variable making changes a tad easier. The NVRAM variable is qos_delay. If QOS-TM is running, it can be shut down with the script traff_off.sh, the NVRAM value set and committed, and the script restarted with traff_on.sh. Data loss would be negligible.

Closely related, data from those executions is saved to regular memory (/tmp/data/traffic.asp) rather than risking wear on the flash memory potentially damaging it. Every so many iterations, it is copied to persistent flash memory. That too is stored in an NVRAM variable qos_cycles. A restart is also required should this value change.

This results in 25 writes/day to flash memory versus the original 86,400.

Of greater interest to me is how often the data is saved to flash memory. That is, should the router reboot unexpectedly, what is the maximum potential data loss? That is calculated by qos_delay times qos_cycles. Using the default settings: 10 seconds x 360 = 3600 seconds = 1 hour. The average loss would be 30 minutes.

NOTE: Running on a UPS, I don't recall the router rebooting unexpectedly in... well ever really. I experience months of consistent up time and eventually will have cause to reboot it myself.

Rebooting is a negligible data loss issue if performed from the DD-WRT GUI (or reboot from Telnet.SSH). In that case, the Shutdown and Startup scripts insure that no more than qos_delay (10 seconds) of data is lost.

The trade-offs become... MyPage refresh time v. data safety v. flash memory wear v. CPU demand.

I did notice an increase in CPU use with the original 1 second sleep time. Thus, the increase to 10 seconds. I haven't seen excessive CPU use since but... I have a relatively speedy Asus RT-N16 router (Broadcom BCM4718 CPU @ 480Mhz), traffic through it hasn't been particularly heavy, and my maximum download speed is "only" 30Mbit.

As we discussed off-board, your Atom CPU solution is vastly preferable. But, for a residential application tracking just a handful of hosts, this has proven more than sufficient thus far.

Thanks.

[alexatkinuk]

The problem I encountered wasn't related to the monitoring script at all, it happens even when its not running.

The problem seems to be that passing the packets through the RRDIPT chain exhausted the software interrupt ability of the routers CPU.

I'm guessing this is due to the fact its not just counting the traffic, its actually routing it around memory to be able to count it and that uses CPU power. Of course, I might be wrong as iptables is pretty much voodoo magic to me.

[mikeones]

Hi, this traffic monitor add-on is awesome! Great work and IMO its a much cleaner solution compared to the other per host traffic monitoring solutions I have tried on wrt.

I have been able to get your work going without much hassle and MyPage.asp is being populated with host data, but when I access the reporting page at http://ip/user/traffic_summary.html, the page is blank except for the summary text title page.

The version I am running is http://dl.dropbox.com/u/6624068/traff_setup.tar.gz and my browser says v0.13 in the tab. Is this the latest version? Any idea why the summary page would be blank?

I noticed these three js files are included.

[Howt]

Thanks. I'm glad you like it. I believe the problem is .../js/traffic.js missing.

That file is created at 11:59pm by the default cron job. My guess is that you simply haven't had that occur yet.

You can force the issue by running the script manually.

[Howt]

The script traff_2js.sh has been updated to correct an error calculating bandwidth. The setup package (see first post) has been corrected. If you have already installed QOS-TM, it may be easier to download the one script which has changed.
http://dl.dropbox.com/u/6624068/traff_2js.sh

EXPLAIN: The error was in calculating bandwidth for the first day of a reporting period. The script assumed the router always reboots at the start of a period and would mark the data as such. Obviously, that is wrong.

[wollyka]

Hi
Thanks for the script but i am having problems running it.
I have a Linksys E4200v1 with DD-WRT v24-sp2 (07/20/12) big - build 19519
I followed the instructions but i am always getting a blank page with going to http://ip/user/traffic_summary.html. I only get the title.
Also i noticed that My_Page is not showing the connections but it is stuck on calculating...

I tried the original script by alexatkinuk and is working fine.. But i need periodic data monitoring with few data loss that your script provides.
Any help is appreciated
Thanks

Edit: the js error i am getting when running the traffic_summary.html report is:

[jozwikjp]

Hello,

I have installed the script and have manually ran the traff_2js.sh script and verified that the process is running.

But I am getting various errors saying "bad number"
here is the output. Could this be a space somewhere or something.

[ebagnoli]

Hi all:

I have tried to follow the tutorial and installed the script in /mnt.

But I have an HDD mounted and not an USB stick. Is it possible to load the script on HDD and what is the syntax?

Tks

[tmphuc]

[staktrace]

Thanks a lot for this, it's something I've been wanting for a while!

Also, just so you know, there is a typo in the traff_install.sh script that caused it to fail for me. The following line:

chmod +x ./$MyPage/*.sh

has an extra $ and really should be this:

chmod +x ./MyPage/*.sh

[z7ths]

All the DropBox links are not working.

does anybody have a copy of this script and could maybe post it / attach it / upload it?

Thank you!

[KittyChampion]