ip_conntrack with QoS and bandwidth use reporting

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Contributions Upload
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Author Message
majorbean
DD-WRT Novice


Joined: 12 Oct 2010
Posts: 4

PostPosted: Tue Jan 22, 2013 3:56    Post subject: Reply with quote
had this working on a Dlink 835 which is Atheros but it never servives a reboot. Damn, wish i bought a Bcm cause bandwidth managment is a neccesity with teenagers nowadays. Plus Telus is invoking restrictions as of Feb. 1. Gonna be an interesting month here in Canada
Sponsor
puck_cent
DD-WRT Novice


Joined: 02 Feb 2007
Posts: 2

PostPosted: Mon Mar 25, 2013 18:59    Post subject: Reply with quote
How long does it take for the stats to initially show up once this is complete? I have successfully added the page to my router, but Clients, Active Protocols, and Active Connections only ever say CALCULATING ...

I have enabled QOS for both WAN and LAN and set the uplink and downlink limits.
DesktopMasters
DD-WRT Novice


Joined: 30 Dec 2011
Posts: 5

PostPosted: Sat Mar 30, 2013 0:44    Post subject: Reply with quote
In IE8 and the current version of chrome I get..

Code:

Port error: Could not establish connection. Receiving end does not exist. miscellaneous_bindings:236
chromeHidden.Port.dispatchOnDisconnect miscellaneous_bindings:236
Uncaught ReferenceError: ASSURED is not defined
(anonymous function)
(anonymous function) qos_conntrack.js:377
request.onreadystatechange common.js:836

_________________
~ Merlin
Falcon4
DD-WRT User


Joined: 26 Sep 2007
Posts: 67
Location: Fresno, CA

PostPosted: Sun Apr 28, 2013 3:10    Post subject: Reply with quote
There's a little bug in traffic_monitor.sh with the latest DD-WRT version. My router's information:
Router Model: Buffalo WZR-HP-AG300H
Firmware Version: DD-WRT v24-sp2 (04/15/13) std - build 21286
Kernel Version: Linux 3.8.7 #224 Mon Apr 15 04:19:40 CEST 2013 mips

Symptom: only the addresses that end in a last-byte beginning in the last-byte of the router IP will be counted. For example, if the router is 192.168.5.1 (as is mine), only addresses 192.168.5.1, 192.168.5.10-19, and 192.168.5.100-199 will be monitored. The rest will not be visible in the MyPage listing.

Cause: A syntax error in the "awk" command in line 4 of traffic_monitor.sh causes it to recognize and store a bad subnet prefix. In the script, mine was stored as, quote, "192.168.5.1." (with a dot at the end). I didn't understand "awk" going into this debugging, but I did a "man" and figured out that "BEGIN" and "END" are needed to initialize the capture (of "nvram get") then process the output at the end - not all at once.

Resolution: In line 4 of traffic_monitor.sh:
Code:
LAN_TYPE=$(nvram get lan_ipaddr | awk ' { FS="."; print $1"."$2 }')
Reformat the awk with beginning and ending brackets:
Code:
LAN_TYPE=$(nvram get lan_ipaddr | awk 'BEGIN { FS = "."; } END { print $1"."$2 }')


Then all addresses are shown. Smile

I was trying to track down a bandwidth hog on my home network, and when I updated the firmware to the latest revision, RFlow was gone - which I just set-up! So I Googled around a bit and found it was removed due to bugs, and I was left without a solution. This page came up but does not provide a download link in the post without being logged-in to this forum! There is literally no attachment visible (or screenshot) on the original post without logging in. Then everything is visible Google users are hence screwed. I think it would be helpful if you could post it somewhere like Dropbox, or post a bold message that we have to log in/register to download the script. Smile

Thanks for this awesome tool! It's definitely the cleanest interface I could imagine for what I want to do with it!

_________________
Falcon4
DD-WRT User


Joined: 26 Sep 2007
Posts: 67
Location: Fresno, CA

PostPosted: Sun Apr 28, 2013 7:06    Post subject: Reply with quote
Further expanding the script:

Get all hostnames (not just static DHCP entries) by reformatting "/tmp/dnsmasq.leases" as a makeshift "/tmp/hosts" file.

Add line above line 49 in traffic_monitor.sh:
Code:
cat /tmp/dnsmasq.leases | awk '{ print $3 " " $4 }' > /tmp/hosts.traf

Change line 27 in qos_conntrack.sh from:
Code:
awk '{ printf "    '\''%s'\'': { online: false, name: \"%s\" },\n",$1,$2; }' /tmp/hosts

to:
Code:
awk '{ printf "    '\''%s'\'': { online: false, name: \"%s\" },\n",$1,$2; }' /tmp/hosts.traf


Then the list will show all DNS names for current leases - much easier to track down computer names!

Also, if I could completely disable the listing of active connections by default, I would. It's giving me a JavaScript error (unknown function "ASSURED") which seems to be related to mis-interpreting the "traffic.asp" fields. Not a JavaScript/object-oriented geek, so I couldn't follow all the parameters around to debug that... I just disabled the update in the code by commenting it out on the router side...

_________________
Falcon4
DD-WRT User


Joined: 26 Sep 2007
Posts: 67
Location: Fresno, CA

PostPosted: Sun Apr 28, 2013 9:05    Post subject: Reply with quote
And a little further, repairing the connection-table parsing and making it "self-aware" of changes to the ip_conntrack syntax.

This is a big one. It replaces the block of "if conntrack = 19 / else" completely, and is a complete rewrite of the code used to generate the ip_conntrack entry in traffic.asp.

Here's the code (with a few original lines above and below):
Code:
  #awk 'BEGIN { printf "{hosts::"} { printf "'\''%s'\'','\''%s'\'',",$1,$2; } END { print "'\''<% show_wanipinfo(); %>'\''}"}' /tmp/hosts >> /tmp/traffic.dat
  cp /proc/net/ip_conntrack /tmp/ipconntrack
  awk 'BEGIN {
    printf "{ip_conntrack::";
  }
  {
    src=""; dst=""; sport=""; dport="";
    for ( x = 2; x <= NF; x++ ) {
      split($x,pairs,"=");
      if (pairs[1] == "src") if (src=="") src=pairs[2];
      if (pairs[1] == "dst") if (dst=="") dst=pairs[2];
      if (pairs[1] == "sport") if (sport=="") sport=pairs[2];
      if (pairs[1] == "dport") if (dport=="") dport=pairs[2];
      if (pairs[1] == "bytes") bytes=pairs[2];
    }
    printf "'\''%s'\'','\''%s'\'','\''%s'\'','\''%s'\'','\''%s'\'',bytes=%s,",$1,src,sport,dst,dport,bytes;
  }
  END {
    print "'\''-'\''}"
  }' /tmp/ipconntrack >> /tmp/traffic.dat
  iptables -L RRDIPT -vnx -t filter | grep ${LAN_TYPE} | awk 'BEGIN { printf "{bw_table::" } { if (NR % 2 == 1) printf "'\''%s'\'','\''%s'\'',",$8,$2; else printf "'\''%s'\'',",$2;}' >> /tmp/traffic.dat


It:
- First makes a temp copy of the ip_conntrack file. Doing awk from a proc file is really sketchy since the "proc" file may change while it's being processed, and screw everything up.
- Uses a for loop to process the parameters *ANYWHERE* they happen to be in the line. The for loop is the first implementation I've seen yet of a "dynamic" field addressing scheme (i.e. "$x" to scan/process each field), so that was fun.
- Uses awk variables to "construct" each connection parameter
- Is intelligent enough to realize that ip_conntrack seems to have multiple "src=" and "dst=" parameters for each connection entry, which defies all laws of programming... weird.
- Builds each connection entry in the traffic.asp file with a fixed set of parameters, so JavaScript parsing errors are all but impossible.

Hope this helps! Smile

_________________
gluon
DD-WRT Novice


Joined: 15 Feb 2013
Posts: 11

PostPosted: Wed Jun 26, 2013 13:38    Post subject: Reply with quote
@Falcon4
Does this script need 'warming up' before displaying any data or really real time ?, mine always shows Calculating while i have checked in top, the script already running, . Would you mind posting your modified files?.

The error i've got is "Uncaught ReferenceError: ASSURED is not defined : MyPage.Asp:1" (chrome), from firefox "ReferenceError: ASSURED is not defined @ http://192.168.1.1/user/qos_conntrack.js:377" referring to this line " eval('setCONTable(' + u.ip_conntrack + ')'); ".



below is the edited files based on your advices and others. Would you mind posting your modified files?
Falcon4
DD-WRT User


Joined: 26 Sep 2007
Posts: 67
Location: Fresno, CA

PostPosted: Fri Jun 28, 2013 5:35    Post subject: Reply with quote
Mine stopped working too - I made some more adjustments then eventually gave up on this script when the router became unstable each day. Running a process in the background for monitoring this information, and creating iptables rules for each connection is an absolutely absurd way to monitor traffic. I just went back to hoping something better will come along soon and turned off the custom page completely... /sigh

Also, the ASSURED error is due to mis-interpreting the fields of the utility output it's using to get its information. In my patched code, there's a more intelligent method of detecting this information, but it's been a while since I looked at it, and I'd suspect your code isn't patched properly to use that new interpreter for it. "ASSURED" is a value in a column right next to the one it's supposed to be reading - so it grabs that value instead of the correct value (which I think is the number of bytes transmitted).

_________________
netomx
DD-WRT Novice


Joined: 06 Oct 2011
Posts: 7

PostPosted: Wed Feb 26, 2014 19:23    Post subject: Reply with quote
@alexatkinuk

BRILLIANT WORK!

Thanks!
mantry
DD-WRT Novice


Joined: 15 Nov 2013
Posts: 3

PostPosted: Wed Mar 05, 2014 16:16    Post subject: Reply with quote
Howdy Guys. I'd like to try this out on my dd-wrt router.

I have a Linksys WRT54G-TM that is running v24-sp2 (07/22/09) mega.

I started down the install but got stuck right quick at the UNZIP command. My firmware does not have the unzip command.

Any suggestions how I get around that? I saw where someone used something move the files across after unzipping them on his windows box.

I have windows, mac os, an FTP site at my ISP, etc at my disposal.

Also, is this software: QoS IP Connection Tracking / Bandwidth Monitor still valid and active or are others using other things? I am mainly interested in tracking bandwidth up/down from my ISP/WAN and seeing which devices on my network are responsible for how much each month.

Thank you in advance!
Quentangle
DD-WRT Novice


Joined: 12 Mar 2014
Posts: 2

PostPosted: Wed Mar 12, 2014 13:59    Post subject: Reply with quote
mantry wrote:
I started down the install but got stuck right quick at the UNZIP command. My firmware does not have the unzip command.

You can install unzip via the opkg command. In order to use that, use the various guides on the wiki.
Official Optware page.
Updated guide.
Optware on Kong's K3-AC-ARM builds.

mantry wrote:
Any suggestions how I get around that? I saw where someone used something move the files across after unzipping them on his windows box.

I have windows, mac os, an FTP site at my ISP, etc at my disposal.

If you don't want to use opkg to install unzip, then you could enable ssh and use scp to transfer the extracted files over. For instance, WinSCP is a nice GUI implementation of the protocol.
Although I suggest that you use the first method, as you want to have usb storage to make the script permanent and it really only takes a few more commands to install unzip from there.
vortex05
DD-WRT User


Joined: 14 Mar 2014
Posts: 183

PostPosted: Sat Mar 15, 2014 20:40    Post subject: Reply with quote
I also get the endless calculating problem I don't know what it is but for me it has something to do with the script trying to start up too early.

Putting a large delay on startup works for me.

Code:

cp -r /jffs/MyPage /tmp/
ln -s /tmp/MyPage/www/qos_conntrack.js /tmp/www/
chmod +x /tmp/MyPage/*.sh
sleep 30
/tmp/MyPage/traffic_monitor.sh <optional refresh interval>&


Ok I couldn't resist I made some adjustments that hopefully will make it better for everyone.

Along with the delay to the startup script I've
changed line 61 of traffic_monitor.sh to:

Code:

REFRESH_RATE="$1"
if [ -z "$REFRESH_RATE" ];
then
  REFRESH_RATE=1
fi


...

sleep $REFRESH_RATE


You can now adjust CPU usage by slowing down the traffic worker's thread in your startup script

Code:

cp -r /jffs/MyPage /tmp/
ln -s /tmp/MyPage/www/qos_conntrack.js /tmp/www/
chmod +x /tmp/MyPage/*.sh
sleep 30
/tmp/MyPage/traffic_monitor.sh 10&


For example this startup script sets the sample time to 10 seconds so it no longer samples super fast eating up CPU cycles.

Code:

cp -r /jffs/MyPage /tmp/
ln -s /tmp/MyPage/www/qos_conntrack.js /tmp/www/
chmod +x /tmp/MyPage/*.sh
sleep 30
/tmp/MyPage/traffic_monitor.sh&



Note: The above will still work as I wanted to make sure the original poster's instructions was backwards compatible it will default to 1 in this situation. So you don't have to update your startup script if you don't want to.

Note this is a modification to version 13c I skipped version 13d because there were some severe bugs in that one.

Rather than get everyone to modify there traffic_monitor.sh I have uploaded mine as version 0.13e for evaluation.

Summary the refresh interval of the traffic monitor now obeys the global refresh interval set for the UI of the router under admin auto-refresh interval.
ronaldol
DD-WRT User


Joined: 28 Apr 2011
Posts: 60

PostPosted: Sat Aug 09, 2014 17:56    Post subject: Buffalo WZR-300HP DD-WRT factory can't download with wget Reply with quote
Hi,

I'm trying to download qos_conntrack_0.13c.zip from csdprojects.co.uk

I have a Buffalo model WZR-300HP DD-WRT factory
Firmware Version DD-WRT v24SP2-MULTI (07/09/12) std - build 19438

When I execute the wget I got the error

root@ATL-FILIAL:/tmp# wget -c http://csdprojects.co.uk/DD-WRT/qos_conntrack_0.13c.zip
Connecting to csdprojects.co.uk (176.126.244.253:80)
wget: not an http or ftp url: https://csdprojects.co.uk/DD-WRT/qos_conntrack_0.13c.zip

Anyone could help me? How could I download?

Regards,

Ronaldo Luiz
alexatkinuk
DD-WRT User


Joined: 07 Dec 2010
Posts: 131
Location: Sheffield, UK

PostPosted: Sat Aug 09, 2014 18:59    Post subject: Reply with quote
Sorry about that, I migrated my whole site over to SSL forgetting all about this.

It should download okay now although bear in mind I cannot provide any support setting it up as I do not use DD-WRT any more.

I have also added the modified 13e from above so that its easier to install if someone wants to use that version instead. That of course would be http://csdprojects.co.uk/DD-WRT/qos_conntrack_0.13e.zip

_________________
2xWZR-HP-G300NH(B) (B0 B0) DD-WRT v24-sp2 (06/14/11) std 17201
One antenna swapped for an RP-SMA connector and 14dB external Yagi.
http://csdprojects.co.uk/ddwrt/
ronaldol
DD-WRT User


Joined: 28 Apr 2011
Posts: 60

PostPosted: Sun Aug 10, 2014 15:39    Post subject: Buffalo WZR-300HP DD-WRT factory ReferenceError: ASSURED is Reply with quote
I'm getting an error...

GET http://100.100.100.64/user/traffic.asp [HTTP/1.0 200 Ok 84ms]
ReferenceError: ASSURED is not defined

Seems to me that is an old error...

I read all replies and didn't find any way to solve it. Any tip of what I need to do?

Regards, Ronaldo Luiz
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next Display posts from previous:    Page 6 of 8
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Contributions Upload All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum