DiFMon: Distributed flow Monitor

A fast, distributed, multipurpose IP flow monitor.

An IP flow is defined as a sequence of packets sharing some properties.
The monitor is in charge of updating a record containing flow-related metrics as soon as a packet is sniffed and collecting all the information of no-more-active flows.
When performing such task, the main issue is represented by the fact that on high speed links the number of active flows is very high (up to hundreds of thousands) and the packet inter-arrival time is very short.
This implies that the time interval spent to search the record associated with a captured packet can be longer than the packet interarrival time, in case of a huge number of flow records.
For this reason we decided to adopt a distribuited approach, the architecture we are developing is composed of three components:

• Meter: which captures the packets arriving at a network interface, associates them to a flow identification number and passes them to next component.
• Flow Cache/s: which store the metrics related to the active flows observed and update the records. This is the most challenging component as it has to search and update the metrics within the interarrival time. For this reason we decided to implement more flow caches as separate processes, and to associate the flows to them on the basis of their identification number. The flow cache is also responsible for "exporting" all the information related to timed-out flows to a further component, i.e. the collector
• Collector: it collects the metrics related to the flows observed by all the flow caches. The stored data can be used by different applications such as traffic profiling, intrusion detection, billing, etc.

The main challenge is the development of a fast, effective flow cache. In particular it is necessary to implement a suitable data structure and ordering mechanism to maintain the information about active flows. We apply an LRU (Least Recently Used) ordering as it is the main solution used in caching algorithms.

The software provides the following capabilties:

• On-line packet sniffing and filtering or packet analisys based on a trace file
• Customizable definition of flows via a scripting language
• Customizable definition of metrics

The whole project is done in C language. We will use the standard libraries, the socket libraries and the libpcap libraries to carry out packet sniffing.
We are developing the software for Linux OS, but we can later provide a Windows version too.

The supported protocols are IP version 4, TCP and UDP. The communication beetween the modules composing the application is carried out by using UDP protocol. However, TCP communications will be implemented too.