Linux Tools for Internet Monitoring?

[James McDonald]

Kirk wrote:
>  
> I have recently put together an Ubuntu (full installation as available 
> online) machine that I have tested and is working.  I have never 
> gotten much past the novice stage with Linux but am competent.  Having 
> said that, I am responsible for an Internet network in three buildings 
> (approx 150 users) and the gateway modem to a VSAT system (1MB 
> uplink/downlink, not shared).  I am not surprised as we have continued 
> to increase bandwidth that it gets sucked up.  I am of the opinion 
> that the nature of Internet behavior and design of websites which 
> include lots of graphics is as much to blame for slow Internet 
> performance as weather conditions or other unknowns that can affect 
> VSAT systems.  My request for advice has to do with what tools are 
> available, if any, on Linux that will allow me to monitor network 
> behavior.  Can I tell if one machine is dominating bandwidth 
> utilization on YouTube, operating Skype or downloading 
> movies/music/porno/whatever?  Can I tell if one of my machines has 
> been co-opted by malicious code and sending out SPAM or anything 
> else?  The VSAT Internet provider will send me very generic graphs on 
> usage but nothing of value.  I would like to be able to respond with 
> more information at hand when I get constant complaints about "the 
> crappy Internet."  If I know that something else is going on, I can 
> pinpoint my efforts at improving how we utilize the bandwidth that is 
> available. 
>  
> I'll take all suggestions and would be happy to contact anyone offline 
> for a more specific conversation.  Thanks.
>  
> Kirk
>  
Kirk,
Some suggestions.

Write a policy stating what the people are allowed to access and what is 
constituted as `fair use'. Gain managerial support for it and get it 
signed off by them. You can then implement controls and point to the 
policy as to why you are doing it when people start moaning about you 
curtailing their freedom to visit tokyochickboy.com.

To get proper monitoring of what is going through your gateway you 
really need to be in between your gateway and your users i.e. all 
traffic is going through your monitoring box. (People will say you can 
do premiscous mode blah blah but with switches these days premiscous 
only gets you broad cast traffic). I use linux as my internet gateway 
boxes and sometimes run iptraf on the external interface to see in real 
time what connections and how many packets are going past.

Implement a transparent proxy using squid and iptables. Use built in 
squid acl rules or dansgaurdian to restrict and control access to 
websites. Install sarg or another squid log monitoring application which 
will tell you daily how much traffic each computer is using for http.

Using firewalling Block ports for many of the known bandwidth hogs 
(streaming radio, bit torrent etc)

Use a quite time on the network to perform benchmark download speed and 
latency tests so you know how quick/slow it can be.

Read the BOFH series for more ideas.