expose /proc/net/dev via lighttpd for off raspap traffic monitoring

[frankozland]

Proposal to expose /proc/net/dev.
It might also be nice to wrap that with charting that can be exposed on raspap dashboard (realtime stats like this would be awesome) as well as exposing the data for off raspberry consumption

I think /proc/net/dev is sort of the universal net traffic data for linux - i need to see traffic on interfaces and confirm that traffic is indeed flowing thru tunnel and at what rate.

This gives users the ability to consume this off raspap for traffic monitor outside of raspap.

this is my crude quick and dirty approach - it would be better to password protect and lock down this stream, or maybe expose with RASPAPI.

The below works for me because i can easily consume this with conky (https://github.com/brndnmtthws/conky)
My firewall rules are tight enuf that im not too worried about hackers.

/etc/lighttpd/conf-enabled $ cat 49-server-stats.conf
server.modules += (
"mod_cgi",
)

$HTTP["url"] =~ "app_serverstats" {
server.document-root = "/var/www/html/serverstats"
alias.url = ( "/app_serverstats" => "/var/www/html/serverstats" )
cgi.assign = ( ".pl" => "/usr/bin/perl" )
server.breakagelog = "/var/log/lighttpd/cgibreakage.log"
}

$cat /var/www/html/serverstats/net_stats.pl
#!/usr/bin/perl
use strict;
use warnings;

print "Content-Type: application/json\r\n\r\n";

Read the /proc/net/dev file

open my $fh, '<', '/proc/net/dev' or die "Cannot open /proc/net/dev: $!";
my @lines = <$fh>;
close $fh;

Parse the data

my %data;
foreach my $line (@lines) {
next if $line =~ /^\sInter|face/; # Skip header lines
if ($line =~ /^\s(\w+):\s*(.*)$/) {
my $iface = $1;
my @values = split(/\s+/, $2);
$data{$iface} = {
receive_bytes => $values[0],
transmit_bytes => $values[8],
};
}
}

Manually construct the JSON string

my $json = "{";
foreach my $iface (sort keys %data) {
$json .= ""$iface":{";
$json .= ""receive_bytes":$data{$iface}->{receive_bytes},";
$json .= ""transmit_bytes":$data{$iface}->{transmit_bytes}";
$json .= "},";
}
$json =~ s/,$//; # Remove the trailing comma
$json .= "}";

Print the JSON string

print $json;

Once i get conky working will post image.

[frankozland]

This is the idea - with this data now exposed from lighttpd, i can easly add a conky indicator (just reads from a /tmp/result.png) generated from python with sparklines for each interface. The interfaces are dynamic - i just chart whatever lighttpd sends.

here is the python script as well...
python_read_server_stats_raspap.py.txt

[frankozland]

final form - the bar on the right is conky. I consume this image (just display the generated image) and now i can see whats going on with raspap without going to a webpage. I ran speed test 3x - its why you see TUN, ETH0 and WLAN0 spike.
I get geoid from ipapi.io poll it every min to my local computer /tmp and just consume it in conky.

[frankozland]

I imagine this python script could be converted to php or javascript+jcanvas, but this was easiest for me. The whole thing could be moved to raspap side, raspap could generate and host the image, and the image could be consumed by admin page, or, if posted on site, anything that has access to the site.

[billz]

@frankozland thanks for the prototype. Not sure if this would supplement or potentially replace the existing dashboard traffic analytics, which does something very similar. Is the idea to make this real time?

pro tip: format code blocks by enclosing them in 3 backticks (```). e.g.:

code goes here

[frankozland]

very close to realtime, yes.

The reason i dropped to iptraf to see network was to prove tun traffic and firewall rules.
The front page graph is nice but it does not document that the traffic is indeed going thru tun - because im so remote - and because of my usecase, i need near live view of the tun traffic to be sure a) the openvpn did not drop (or explain why network slow down) and b) to be certain tun is actually active.

If i switch interfaces, i want again to see that network flow.

I looked at raspapi - i see a lot of stuff is being exposed thru that interface. Im guessin this might be better served putting it in that framework so the data is at least protected with a key. But if doing that, it would make sense to me to host both the raw data and the image result.

For now what i have here exactly serves my use case - i think its likely beyond most users needs, but the blinky lights are nice =)