ipt_netflow

Name

ipt_netflow — Script and kernel modules to turn system into Netflow source.

Synopsis

Description

The ipt_netflow script is used to simplify the process of turning a NST system into a Netflow source (similar to a Cisco router or running rflow on a DD-WRT router). When installed and enabled, your NST system will provide network traffic information via UDP packets to a Netflow collector. The collector can be running on your NST system or on a remote system.

There are several reasons why you might want to turn your NST system into a Netflow source:

Your NST system is serving as a router within your network and you want to provide network traffic information to a external collector at a minimum load to your NST system.
Your NST system is being used as a dedicated server and you would like to be able to track and monitor connection information using another system.
You are developing a Netflow client and you need a test system to act as a Netflow source (to provide data while testing).

There are many Netflow clients which can process data from a Netflow source (ntop, nfcapd, ...). During (or after installation), it is possible to configure the ipt_NETFLOW kernel module to feed data to a Netflow client application.

This script provides several modes of operation which are controlled by the "--mode MODE" command line argument. Here is a brief overview of the available modes (details for each mode can be found in the following sections of this document):

Show (--mode show): The show mode is used to display the current status and configuration of the ipt_NETFLOW kernel module (if it is loaded).
Status (--mode status): The status mode is typically used by scripts to determine if the ipt_NETFLOW kernel module has been loaded. This mode does not display anything to the console unless the -v (verbose) option is specified.
Install (--mode install): The install mode is used to build the ipt_KERNEL module, install and update the associated system files. This mode prepares the system to act as a Netflow source but does not immediately start sending Netflow data. This mode does not display anything to the console unless the -v (verbose) option is specified.
Enable (--mode enable): The enable mode is used to build the ipt_KERNEL module, install and update the associated system files, and load the ipt_KERNEL module. The system will be acting as a Netflow source after enabling. However, you should refer to the installation section of this document as you will need to manually adjust some files if you want the system to behave as a Netflow service at boot time. This mode does not display anything to the console unless the -v (verbose) option is specified.
Disable (--mode disable): The disable mode is used to remove the NETFLOW rules from the active iptables and to unload the ipt_KERNEL module. It does not remove any of the associated system files. This mode does not display anything to the console unless the -v (verbose) option is specified.
Remove (--mode remove): The remove mode is similar to the disable mode except that it does remove the associated system files. This mode does not display anything to the console unless the -v (verbose) option is specified.

A ipt_netflow service is included to simplify the usage. When the ipt_netflow service is used, you will not need to run this script by hand.

The ipt_netflow Service

The simplest method for setting up your system to act as a Netflow source does not require you to run this script directly. Instead, you use the ipt_netflow service.

Edit the /etc/sysconfig/ipt_netflow service configuration script.
Use the systemctl command to enable the ipt_netflow service to run when the system boots.
Also use the systemctl command to directly manage the ipt_netflow service.

The following provides an example of the commands you will run to edit, enable and start the ipt_netflow service:

[root@probe ~]# vi /etc/sysconfig/ipt_netflow
[root@probe ~]# systemctl enable ipt_netflow.service
[root@probe ~]# systemctl start ipt_netflow.service
[root@probe ~]# systemctl status ipt_netflow.service
ipt_netflow.service - Netflow IPTables Service
   Loaded: loaded (/usr/lib/systemd/system/ipt_netflow.service; enabled)
   Active: active (exited) since Mon 2013-09-23 07:57:16 EDT; 21s ago
  Process: 5724 ExecStart=/usr/share/ipt_netflow/systemd/ipt-netflow_exec (code=exited, status=0/SUCCESS)

Sep 23 07:57:16 shopper2 systemd[1]: Starting Netflow IPTables Service...
Sep 23 07:57:16 shopper2 systemd[1]: Started Netflow IPTables Service.
Sep 23 07:57:16 shopper2 ipt-netflow_exec[5724]: Starting Netflow service (loading ipt_netflow module): [  OK  ]
[root@probe ~]#

Installation

Here is a example of using the ipt_netflow script to build and install the ipt_NETFLOW kernel module and prepare the sysem so that it will be ready to send Netflow data to port 2055 on the 192.168.1.100 system:

[root@probe ~]# ipt_netflow -m install --destination "192.168.1.100:2055"
[root@probe ~]#

There are numerous parameters which you can adjust for the ipt_NETFLOW kernel module. Other than the destination parameter, you may not need to worry too much about them unless your NST system is acting as a server or router for a large amount of traffic. You can adjust these kernel module parameters using three different methods:

Initially when you use this script to do your Netflow installation (in either "install" or "enable" mode). Refer to the Options section below for details on the command line arguments that can be used to configure the installation.
You can use the sysctl command to temporarily adjust the ipt_NETFLOW kernel module configuration. These are "on-the-fly" modifications and will be lost when the ipt_NETFLOW kernel module is unloaded.
For permanent modification after installation, you should edit the file: "/etc/modprobe.d/ipt_netflow.conf". The options specified in this file will be applied whenever the ipt_NETFLOW kernel module is loaded.

The following provides a summary of the ipt_NETFLOW kernel module parameters which can be adjusted. These parameters can be set when using "install" or "enable" modes. Details about these parameters are brief or omitted as this is just a summary. If you need more details, you should extract the ipt_NETFLOW kernel module source code from the tar file found under the /usr/share/ipt_netflow directory.

destination

The destination parameter is used to specify where the Netflow data is to be sent. The Netflow data can be sent to 0 or more UDP addresses. For example, to send the data to port 2055 on the 192.168.1.136 system and port 5055 on the 192.168.1.10 system, you would specify a value like:

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... destination=192.168.1.136:2055,192.168.1.10:5055 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.destination="192.168.1.136:2055,192.168.1.10:5055"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install -destination "192.168.1.136:2055,192.168.1.10:5055" ...
#
# OR:
#
# ipt_netflow -m enable -destination "192.168.1.136:2055,192.168.1.10:5055" ...

active_timeout

The active_timeout parameter is used to control how often active connection information is flushed out (at least that's our interpretation). If not specified, the default value of 1800 seconds (30 minutes) will be used. For example, to set a "15" second time out, you would specify:

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... active_timeout=15 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.active_timeout="15"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --active-timeout "15" ...
#
# OR:
#
# ipt_netflow -m enable --active-timeout "15" ...

inactive_timeout

The inactive_timeout parameter is used to control how often inactive connection information is exported. If not specified, the default value of 15 seconds will be used. For example, to set a "20" second time out, you would specify:

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... inactive_timeout=20 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.inactive_timeout="20"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --inactive-timeout "20" ...
#
# OR:
#
# ipt_netflow -m enable --inactive-timeout "20" ...

aggregation

The aggregation parameter is used to reduce (collapse) how much information is sent. If you need to use this parameter, you should review the source code for the ipt_NETFLOW kernel module (the README files might prove useful). For example, the following demonstrates how to set this parameter to a value of "192.0.0.0/8=16,80-89=80":

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... aggregation=192.0.0.0/8=16,80-89=80 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.aggregation="192.0.0.0/8=16,80-89=80"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --aggregation "192.0.0.0/8=16,80-89=80" ...
#
# OR:
#
# ipt_netflow -m enable --aggregation "192.0.0.0/8=16,80-89=80" ...

debug

The debug parameter is used to set the debug level of the ipt_NETFLOW kernel module. We are not entirely sure where this debug information appears. The following shows how to set the debug level to a value of "1":

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... debug=1 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.debug="1"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --debug "1" ...
#
# OR:
#
# ipt_netflow -m enable --debug "1" ...

maxflows

The maxflows parameter is used to limit the number of flows that will be tracked. It defaults to 2000000 if not specified. Specify a value of 0 to remove the limit, but be careful as this limit is used to prevent DOS attacks. The following shows how to set this value to "1000000":

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... maxflows=1000000 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.maxflows="1000000"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --maxflows "1000000" ...
#
# OR:
#
# ipt_netflow -m enable --maxflows "1000000" ...

hashsize

The hashsize parameter is used when trying to optimize the size of the hash table used by the ipt_NETFLOW kernel module. It defaults to 0 if not specified in which case the module will automatically pick a size based on the amount of available memory. For performance tuning it should be set to a value approximately twice the size of the number of flows which you normally have. The following demonstrates how you could specify a value of "15000":

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... hashsize=15000 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.hashsize="15000"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --hashsize "15000" ...
#
# OR:
#
# ipt_netflow -m enable --hashsize "15000" ...

sndbuf

The sndbuf parameter is used to set the size of the send buffer. It defaults to 0 if not specified in which case the module will automatically pick a size. If you notice Netflow packets being dropped (look for the "sock: fail" value in /proc/net/stat/ipt_netflow), then you will probably need to increase this value. The following demonstrates how you would specify a value of "128000":

# As it would appear on the options line in /etc/modprobe.d/ipt_netflow.conf
#
options ipt_NETFLOW ... sndbuf=128000 ...

# For sysctl at the command line you would use
#
# sysctl -w net.netflow.sndbuf="128000"

# For setting at the time of script invocation, you would use:
#
# ipt_netflow -m install --sndbuf "128000" ...
#
# OR:
#
# ipt_netflow -m enable --sndbuf "128000" ...

Enabling

Here is a example of using the ipt_netflow script to build and install the ipt_NETFLOW kernel module such that the system will start sending Netflow data immediately to port 2055 on the 192.168.1.100 system:

[root@probe ~]# ipt_netflow -m enable --destination "192.168.1.100:2055"
[root@probe ~]#

Using "enable" mode is almost identical to using "install" mode. The difference is that "enable" mode will immediately attempt to load the ipt_NETFLOW kernel module and update your iptable rules to enable the sending of Netflow data. If you have a custom iptables configuration, you should use "install" mode instead and customize your iptables configuration and rules files manually.

iptables Configuration

After using the ipt_netflow script to install and enable the ipt_NETFLOW module, the system will be a functioning Netflow source. However, it will not be permanently configured. This means if you reboot your system, it will not automatically come up enabled as a Netflow source. If you want to make your Netflow installation permanent you will need to make some changes to iptables. In particular, you will need to:

Make sure the iptables service is enabled. This is done via:

[root@probe ~]# systemctl start iptables.service
[root@probe ~]#

Make sure the /etc/sysconfig/iptables-config file is updated to load the ipt_NETFLOW kernel module. This is done by editing the file and adding the ipt_NETFLOW module to the IPTABLES_MODULES list. For example, the modified line might look like:
```
#
# Example IPTABLES_MODULES setting when only the ipt_NETFLOW module is required
#
IPTABLES_MODULES="ipt_NETFLOW"
```

Make sure the /etc/sysconfig/iptables file has NETFLOW rules inserted in the proper location. If you don't have a /etc/sysconfig/iptables rules file yet, you should be able to create one with the NETFLOW rules already inserted using the following commands:

[root@probe ~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]
[root@probe ~]# cat /etc/sysconfig/iptables
# Generated by iptables-save v1.4.7 on Thu Mar  3 10:03:22 2011
*filter
:INPUT ACCEPT [1361:200518]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [579:133576]
-A INPUT -j NETFLOW 
-A FORWARD -j NETFLOW 
-A OUTPUT -j NETFLOW 
COMMIT
# Completed on Thu Mar  3 10:03:22 2011

[root@probe ~]#

Warning

If you have complex iptables rules, DO NOT use the steps above to modify your iptables rules file. You will need to decide where to insert the "-j NETFLOW" rules based on the requirements of your network.

Show

Once installed, you can use invoke this script with the "--mode show" option to get a detailed report on the system files which have been created by the script, the current configuration parameters for the ipt_NETFLOW kernel module, and the current status of the ipt_NETFLOW kernel module. Here is a example:

[root@probe ~]# ipt_netflow -m show

The following lists the system files created by this script which
are used to configure, load and use the ipt_NETFLOW kernel module:

-rw-r--r-- 1 root root    220 Mar  3 06:51 /etc/modprobe.d/ipt_netflow.conf
-rw-r--r-- 1 root root 310503 Mar  3 06:51 /lib/modules/2.6.34.7-66.fc13.i686/extra/ipt_NETFLOW.ko
-rwxr-xr-x 1 root root   4816 Mar  3 06:51 /lib/xtables/libipt_NETFLOW.so

The following shows the current sysctl parameters related to the
ipt_NETFLOW kernel module. You can adjust them on the fly using the
sysctl command. To make permanent changes, edit the
/etc/modprobe.d/ipt_netflow.conf file.

net.netflow.active_timeout = 1800
net.netflow.inactive_timeout = 15
net.netflow.debug = 0
net.netflow.hashsize = 12287
net.netflow.sndbuf = 112640
net.netflow.destination = 127.0.0.1:2055
net.netflow.aggregation = 
net.netflow.maxflows = 2000000

The following shows the contents of /proc/net/stat/ipt_netflow which contains
the current ipt_NETFLOW kernel module status:

Flows: active 7 (peak 8 reached 0d0h25m ago), mem 0K
Hash: size 12287 (mem 47K), metric 1.0, 1.0, 1.0, 1.0. MemTraf: 176 pkt, 21 K (pdu 0, 0).
Timeout: active 1800, inactive 15. Maxflows 2000000
Rate: 7192 bits/sec, 5 packets/sec; Avg 1 min: 4611 bps, 2 pps; 5 min: 1922 bps, 0 pps
cpu#  stat: <search found new, trunc frag alloc maxflows>, sock: <ok fail cberr, bytes>, traffic: <pkt, bytes>, drop: <pkt, bytes>
Total stat:      0    566    343,    0    0    0    0, sock:     59 0 59, 17 K, traffic: 909, 0 MB, drop: 0, 0 K
sock0: 127.0.0.1:2055, sndbuf 112640, filled 1, peak 1; err: sndbuf reached 0, other 0


[root@probe ~]#

Status

Use "--mode status" to determine if the ipt_NETFLOW kernel module is loaded and whether the system is currently acting like a Netflow server. This is mode is typically used by other scripts as it does not produce any output unless you include the "--verbose" (or -v) option. Here is a example:

[root@probe ~]# ipt_netflow -m status && echo "Netflow OK"
Netflow OK
[root@probe ~]#

Disable

Use "--mode disable" to temporarily disable the system from acting like a Netflow server (rules will be removed from iptables and the ipt_NETFLOW kernel module will be unloaded. This is mode will not produce any output unless you include the "--verbose" (or -v) option. Here is a example:

[root@probe ~]# ipt_netflow -m disable && echo "Netflow DISABLED"
Netflow DISABLED
[root@probe ~]#

Remove

Use "--mode remove" to permanently disable the system from acting like a Netflow server. Rules will be removed from iptables and the ipt_NETFLOW kernel module will be unloaded. In addition, system files will be removed and/or updated to permanently disable Netflow. This is mode will not produce any output unless you include the "--verbose" (or -v) option. Here is a example:

[root@probe ~]# ipt_netflow -m remove && echo "Netflow REMOVED"
Netflow REMOVED
[root@probe ~]#

Options

The following command line options are available:

[-m TEXT] | [--mode TEXT]: This option controls what ipt_netflow will do. If you specify "status", the script will exit with a 0 return code if the system appears to have the ipt_NETFLOW module installed and configured in iptables (include -v if you want ASCII output in addition to the exit code). If you specify "show" (the default), the script will check to see if the ipt_NETFLOW module is loaded and then show the current settings and status. If you specify "install" the script will build, but not load the necessary kernel module to enable Netflow output. If you specify "enable" the script will build and load the necessary kernel module to enable Netflow output. Rules will also be added to your iptables to enable the system to output Netflow information. If you specify "disable" it will update your iptables rules (attempt to remove any NETFLOW entries) and then remove the ipt_NETFLOW kernel module. If you specify "remove" it will disable the Netflow configuration and remove any kernel modules or libraries which were installed.
[-d TEXT] | [--destination TEXT]: This option is only used in "install" mode. It sets the default destination list for the Netflow UDP packets. If omitted, it defaults to "127.0.0.1:2055". You can specify more than one destination. For example: "127.0.0.1:2055,192.168.1.136:2055". You can also view and change the destination list on the fly through the use of the "net.neflow.destination" sysctl parameter.
[--active-timeout INTEGER]: This option corresponds to the "active_timeout" module parameter recognized by the ipt_NETFLOW kernel module. It is specified in seconds and defaults to 1800 seconds (30 minutes) if omitted. This value can be changed on the fly using the sysctl command to adjust the net.netflow.active_timeout parameter. The minimum value permitted is 1. The maximum value permitted is 86400.
[--inactive-timeout INTEGER]: This option corresponds to the "inactive_timeout" module parameter recognized by the ipt_NETFLOW kernel module. It is specified in seconds and defaults to 15 seconds if omitted. This value can be changed on the fly using the sysctl command to adjust the net.netflow.inactive_timeout parameter. The minimum value permitted is 1. The maximum value permitted is 86400.
[--debug INTEGER]: This option corresponds to the "debug" module parameter recognized by the ipt_NETFLOW kernel module. It is typically left at 0 to disable debug output. This value can be changed on the fly using the sysctl command to adjust the net.netflow.debug parameter. The minimum value permitted is 0.
[--hashsize INTEGER]: This option corresponds to the "hashsize" module parameter recognized by the ipt_NETFLOW kernel module. It defaults to 0 if not specified in which case the module will automatically pick a size based on the amount of available memory. For performance tuning it should be set to a value approximately twice the size of the number of flows which you normally have. This value can be changed on the fly using the sysctl command to adjust the net.netflow.hashsize parameter. The minimum value permitted is 0.
[--sndbuf INTEGER]: This option corresponds to the "sndbuf" module parameter recognized by the ipt_NETFLOW kernel module. It defaults to 0 if not specified in which case the module will automatically pick a size. If you notice netflow packets being dropped (look for the "sock: fail" value in /proc/net/stat/ipt_netflow), then you will probably need to increase this value. This value can be changed on the fly using the sysctl command to adjust the net.netflow.sndbuf parameter. The minimum value permitted is 0.
[--maxflows INTEGER]: This option corresponds to the "maxflows" module parameter recognized by the ipt_NETFLOW kernel module. It defaults to 2000000 if not specified. Specify a value of 0 to remove the limit, but be careful as this limit is used to prevent DOS attacks. This value can be changed on the fly using the sysctl command to adjust the net.netflow.maxflows parameter. The minimum value permitted is 0.
[--aggregation TEXT]: This option corresponds to the "aggregation" module parameter recognized by the ipt_NETFLOW kernel module. By default, no aggregation is done. This value can be changed on the fly using the sysctl command to adjust the net.netflow.aggregation parameter.
[-h [true]|false] | [--help [true]|false]: When this option is specified, ipt_netflow will display a short one line description of ipt_netflow, followed by a short description of each of the supported command line options. After displaying this information ipt_netflow will terminate.
[-H [true]|false] | [--help-long [true]|false]: This option will attempt to pull up additional ipt_netflow documentation within a text based web browser. You can force which browser we use setting the environment variable TEXTBROWSER, otherwise, we will search for some common ones.
[-v [true]|false] | [--verbose [true]|false]: When you set this option to true, ipt_netflow will produce additional output. This is typically used for diagnostic purposes to help track down when things go wrong.
[--version [true]|false]: If this option is specified, the version number of the script is displayed.

Files

/etc/sysconfig/ipt_netflow: The configuration file used by the ipt_netflow service.
/var/log/ipt_netflow.log: A short log file containing diagnostic information from the last time the ipt_netflow service was started or stopped.
/lib/modules/$(uname -r)/extra/ipt_NETFLOW.ko: The kernel module that will be built during installation.
/etc/modprobe.d/ipt_netflow.conf: The file which contains the options for the ipt_NETFLOW kernel module. You can edit this file if you need to make permanent adjustments to your configuration. If you only want to experiment with new settings, use the sysctl option instead. NOTE: This file is overwritten by the ipt_netflow service, so don't bother editing it unless you plan on running the ipt_netflow script by hand.
/lib/xtables/libipt_NETFLOW.so or /lib64/xtables/libipt_NETFLOW.so: The NETFLOW library which will be loaded by iptables when NETFLOW rules are processes.
/proc/net/stat/ipt_netflow: This proc entry can be used to get status information related to the ipt_NETFLOW kernel module.
/usr/share/ipt_netflow: Directory containing resource files used by the ipt_netflow script. In particular, the tar ball containing the source code for the ipt_NETFLOW kernel module and library can be found here. You should extract this file and review its contents if you need additional information related to the configuration of the ipt_NETFLOW kernel module.

Environment

TEXTBROWSER: This controls what text based browser is used to display help information about the script. If not set, we will search your system for available text-based browsers (Ex: elinks, lynx ...).