YAMon v4... usage totals by device

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> General Questions
Goto page Previous  1, 2, 3, 4 ... 14, 15, 16  Next
Author Message
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Nov 28, 2019 22:26    Post subject: Reply with quote
I am beginning to make real progress on the long promised, and frequently asked for, `access restrictions` functionality. In a nutshell, a modification to YAMon that will allow you to block internet access to devices at certain times (e.g., after 10pm on school nights).

It's taken a long while because it has to involve functionality on the router, in the reports and also on the usage-monitor.com server.

One of the first features I added in the YAMon reports was the ability to organize your devices into arbitrary logical groupings - e.g., by person or device location or function.
For some time now, things have been organized on the router so that it is easy to add/modify iptables rules which restrict access to the web for specific device groups. What was lacking was the ability to communicate updates in the reports back to the scripts that run on the router (because the standard web servers in the router firmware do not support cgi/bin functionality).

To get around that, I've had to add bits to route the updates through the usage-monitoring.com web site (where you will be able to setup the access restrictions) and then modify the scripts on the router to periodically check for updates.

The interface to add/edit the access restrictions is beginning to take shape at https://usage-monitoring.com/current/SetAccessRestrictions.php

Let me know what you think.

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
Sponsor
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1404

PostPosted: Wed Dec 18, 2019 22:42    Post subject: Reply with quote
al_c wrote:


One of the first features I added in the YAMon reports was the ability to organize your devices into arbitrary logical groupings - e.g., by person or device location or function.

Let me know what you think.

Al


Will this be stored in the local users.js?

_________________
Segment 1 R9000 10Gb LAN, 1Gb ISP BS
Test Bed R9000 10Gb LAN
Wired AP 1 Unifi UAP AC PRO 1Gb LAN
Wired AP 2 Unifi UAP AC PRO 1Gb LAN
Wired AP 3 Unifi UAP AC PRO 1Gb LAN
Test Bed R7800 1Gb LAN OpenWRT Kong
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
www.desipro.de/openwrt/ (Kong's)
YAMon
https://usage-monitoring.com/index.php
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Thu Dec 19, 2019 14:54    Post subject: Reply with quote
ddaniel51 wrote:
al_c wrote:


One of the first features I added in the YAMon reports was the ability to organize your devices into arbitrary logical groupings - e.g., by person or device location or function.

Let me know what you think.

Al


Will this be stored in the local users.js?

short answer: Yes!

longer answer - for the most part but ATM not as before.

In v3 I tried to keep a unique entry for each MAC address... this sometimes meant a lot of thrashing in the file if two devices were competing for the same IP or constantly alternating their IP (as was sometimes the case with some Apple devices). Things could get out of sync when this happened so I needed a *lot* of overhead to correctly update users.js and more importantly to record the hourly & monthly data for the devices (because data is obtain from the IP address).

In v4, I simplified things greatly... I now identify devices and record traffic by the combined MAC & IP addresses so if devices can't make up their minds about IP addresses, there will be multiple entries the data files. It is actually easier for me to consolidate those entries in the reports than on the router (in JS I am able to use more complex data structures which are not possible in the bash scripting environment).

Because a device can now have multiple entries in user.js, I have (temporarily at least) moved the group name to a different data structure (which lists just the MAC and the group name)... this means less updating if/when grouping changes.

So... back to what gets stored in users.js...
When a new device is detected on the network, new entries are created in users.js
1. an entry mapping the MAC address to the `unknown` group
2. entries for the MAC & IP address, the device name (if available) & some timestamps (there couple be a couple if you have IPv6 enabled)

If you have the optional database integration feature turned on, your group entries in users.js will be synced with what you have entered in the report. This will happen automatically whenever the router runs the `check-network.sh` script (i.e, when YAMon starts and/or when changes are detected on the network - i.e., changes in the results of the arp or ip calls).

ATM, I am not automatically syncing the device names in users.js... my rationale for this is that in most cases, the device name is unique to the MAC but sometimes it is not (e.g., if you are running virtual machines on a device so the MAC could have several IPs each with a different name). The scripts running on the router really don't care about the names of the devices... but for the new access restrictions, the groupings are essential.

It would be possible to merge the group name back into the device entries...

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1404

PostPosted: Fri Dec 20, 2019 2:57    Post subject: Reply with quote
al_c wrote:

It would be possible to merge the group name back into the device entries...

Al


Merging the group name into the device entries would make managing a large number of groups and devices like I have much easier to organize and manage. On a local level anyway.

_________________
Segment 1 R9000 10Gb LAN, 1Gb ISP BS
Test Bed R9000 10Gb LAN
Wired AP 1 Unifi UAP AC PRO 1Gb LAN
Wired AP 2 Unifi UAP AC PRO 1Gb LAN
Wired AP 3 Unifi UAP AC PRO 1Gb LAN
Test Bed R7800 1Gb LAN OpenWRT Kong
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
www.desipro.de/openwrt/ (Kong's)
YAMon
https://usage-monitoring.com/index.php
ddaniel51
DD-WRT Guru


Joined: 19 Feb 2013
Posts: 1404

PostPosted: Sun Dec 22, 2019 16:40    Post subject: Reply with quote
Any edits I make via the Device page eventually vanish as the changes revert to where I started.

Local users js is not changed.

_________________
Segment 1 R9000 10Gb LAN, 1Gb ISP BS
Test Bed R9000 10Gb LAN
Wired AP 1 Unifi UAP AC PRO 1Gb LAN
Wired AP 2 Unifi UAP AC PRO 1Gb LAN
Wired AP 3 Unifi UAP AC PRO 1Gb LAN
Test Bed R7800 1Gb LAN OpenWRT Kong
download1.dd-wrt.com/dd-wrtv2/downloads/betas/ (Brain Slayer)
www.desipro.de/openwrt/ (Kong's)
YAMon
https://usage-monitoring.com/index.php
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Mon Dec 30, 2019 21:23    Post subject: Dark Mode has been added to YAMon 4 Reply with quote
So the first iteration of Dark Mode is available in YAMon v4 (yes, you must be running the new version to get this feature)

If you already have v4, you will have to update your yamon4.0.html file (which normally lives in /opt/YAMon4/www/). To get the latest version of this file, go to https://usage-monitoring.com/installv4.php. Alternatively, re-run install.sh (On the plus side, this'll make sure that all of your files are up-to-date)

Once you have the new file in place, you'll find on the `Settings` tab a new 'Dark Mode' checkbox.

The CSS that control the background colours for dark mode can be found at the bottom of yamon4.0.css (https://usage-monitoring.com/current/css/yamon4.0.css). If you want to tweak the settings, copy that text into custom.css on your router (by default, in /opt/YAMon4/www/css/) and have fun. Please share your preferences so that we can make this better for everyone.

Thx!

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Fri Jan 03, 2020 16:43    Post subject: Hmmm. Should I give this a try? Reply with quote
I installed YaMON 3.4.7 yesterday without realizing this 4.04 was available.

The beta has been out awhile now. Is this getting to the point we can depend on it for our reports?
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Fri Jan 03, 2020 17:03    Post subject: Re: Hmmm. Should I give this a try? Reply with quote
croweater wrote:
I installed YaMON 3.4.7 yesterday without realizing this 4.04 was available.

The beta has been out awhile now. Is this getting to the point we can depend on it for our reports?

I just pushed v4.0.5 to the server... lots of little fixes (and one or two major). I think I'm about ready to finally drop the Beta designation and say it is ready to go. Feedback?

see https://usage-monitoring.com/whatsnew.php for a summary of the changes

Re-run install.sh on your router to get the latest and greatest files or go to https://usage-monitoring.com/installv4.php

If you find any issues, please include screenshots and log files and all sorts of info that'll help me find/fix the problem.

Thanks for your continuing support and assistance.

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Fri Jan 03, 2020 22:49    Post subject: Reply with quote
Well this was a mistake.

I ran the install.sh assuming the script would take care of the upgrade for me?

I took all the defaults and it seemed as though the script would handle everything, but it didn't because even after a reboot I'm still running the old version.

I guess this only works on systems that arent already running a previous version?


Last edited by croweater on Sat Jan 04, 2020 1:49; edited 1 time in total
BR07H3R
DD-WRT User


Joined: 24 Dec 2017
Posts: 135

PostPosted: Sat Jan 04, 2020 1:24    Post subject: Reply with quote
If you look on the thread for 3.4 you will see my screenshot still not working here.
Btw I had to do some voodoo to get just to that point.
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Sat Jan 04, 2020 2:47    Post subject: Reply with quote
croweater wrote:
Well this was a mistake.

I ran the install.sh assuming the script would take care of the upgrade for me?

I took all the defaults and it seemed as though the script would handle everything, but it didn't because even after a reboot I'm still running the old version.

I guess this only works on systems that arent already running a previous version?



I apologize for the grief but do not know why things did not work properly at your end. It *most* certainly works on (at least some) systems running v3... I've run the installer repeated under both DD-WRT and Turris Omnia with out issues. What do you mean you are running the old version?!? YAMon 3 & 4 can run simultaneously (as explained previously so that I/you can validate the numbers in the new system.

Did you get the new installer from https://usage-monitoring.com/current/YAMon4/install%20(4.0.5).zip?t=123 ?
Did the files and folders get created properly?(see the install validation suggestions at https://usage-monitoring.com/v4-beta.php)

Are you getting error messages on the screens or in the logs? If yes, please help me resolve the issues by including as much info as possible (screenshots, log files. etc.). Once I know what is going wrong on your system, I can usually fix the issue quite quickly. In the meantime, I'm flying blind.

Thank you in advance for your help.

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Sat Jan 04, 2020 3:27    Post subject: Reply with quote
Okay

I have 2 directories now
/opt/YAMon3
/opt/YAMon4

I got zero errors during the install of 4.0.5 but when I ran the report I noticed it said 3.4.7 at the top.

I looked in the startup and shutdown Command Tab and it still had the ones there for YAMon3. I went ahead and removed those commands and then executed /opt/YAMon4/start.sh

There were no errors but I cant display any of the report data

At this point it is possible all the data is being collected correctly for YAMon4, but I cant figure out how to display the reports so I restarted YAMon3 until this gets resolved.

Let me know if you want me to try anything. Happy to help.


Last edited by croweater on Sat Jan 04, 2020 3:49; edited 1 time in total
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Sat Jan 04, 2020 3:45    Post subject: Reply with quote
croweater wrote:
Okay

I have 2 directories now
/opt/YAMon3
/opt/YAMon4

I got zero errors during the install of 4.0.5 but when I ran the report I noticed it said 3.4.7 at the top.

I looked in the startup and shutdown Command Tab and it still had the ones there for YAMon3. I went ahead and removed those commands and tried to run /opt/YAMon4/start.sh

At this point it is possible all the data is being collected correctly for YAMon4, but I cant figure out how to display the reports so I restarted YAMon3 until this gets resolved.

Let me know if you want me to try anything. Happy to help.

It sounds like your symlinks were not created properly. See the validation steps at https://usage-monitoring.com/v4-beta.php

Try running /opt/YAMon4/start.sh again if they were not created.

If you have configured the aliases, you can restart YAMonv4 by simply entering `sta` in a PuTTY window. For the aliases, add the line
Code:
cp /opt/YAMon4/alias.sh $HOME/.profile ; . $HOME/.profile
to my startup scripts in the DD-WRT admin GUI

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
croweater
DD-WRT User


Joined: 28 Dec 2019
Posts: 59

PostPosted: Sat Jan 04, 2020 4:59    Post subject: Reply with quote
BTW, I'm using DD-WRT. And I am using ssh to login to the router as root.

It looks like all the alias are getting loaded.

When I run /opt/YAMon4/start.sh it executes with no errors.

When I attempt to run the reports it keeps loading... never completes. And I get this error:

Ah nuts! An unexpected JavaScript error has prevented the reports from opening properly.
al_c
DD-WRT Guru


Joined: 13 Apr 2013
Posts: 2134
Location: Ottawa Canada

PostPosted: Sat Jan 04, 2020 5:24    Post subject: Reply with quote
croweater wrote:
BTW, I'm using DD-WRT.

It looks like all the alias are getting loaded.

When I run /opt/YAMon4/start.sh it executes with no errors.

When I attempt to run the reports it keeps loading... never completes. And I get this error:

Ah nuts! An unexpected JavaScript error has prevented the reports from opening properly.


OK - so now you have to open the browser console window... please tell me you are using Firefox or Chrome.

In FF/Chrome, type <ctrl><shift>J to open the browser console. I know that MSIE & Edge also have a console but I do not use either browser so I have no idea of how to open it.

Once the console is open, click the filters at the top right of the window so that only `Errors` is highlighted and then look for files named yamon4.0.js or util4.0.js (in the right most column). Send me a screen shot of that window.

Al

_________________
Thanks to everyone for supporting YAMon!

If you use and like YAMon and have not done so recently, please consider making a donation - thanks!
YAMon GitHub Repository
Goto page Previous  1, 2, 3, 4 ... 14, 15, 16  Next Display posts from previous:    Page 3 of 16
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> General Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum