Using NetFlow/IPFIX for Malware Detection

NetFlow is a network protocol developed by Cisco for collecting IP traffic information and monitoring network traffic. By analyzing flow data, a picture of network traffic flow and volume can be built. Using a NetFlow collector and analyzer, you can see where network traffic is coming from and going to and how much traffic is being generated.

Netflow

A NetFlow collector is a router, probe or software-based collector that generates NetFlow records. The network traffic that have been recorded are exported from the router and collected using a NetFlow collector. A NetFlow collector contains a process which allows the captured traffic to be translated to user-friendly formats which allow network traffic analysis.

IPFIX

IPFIX stands for IP Flow Information Export, IPFIX is very similar to Netflow, in the sense that it allows network engineers and administrators to collect flow information from Switches, Routers and any other network devices that support the protocol. In order it again allows the operator to analyze the Traffic Flow information that is being sent by processing it through a Network/Netflow Analyzer.

IPFIX has been derived from Netflow Version 9, it uses many of the same procedures for Exporting a “flow” to a Collector, which operate in a many-to-many relationship.

Vendors using IPFIX

  • Avaya
  • Barracuda Networks
  • Blue Coat
  • Cisco Systems
  • Citrix
  • Ecessa
  • Extreme Network
  • F5 Networks
  • Juniper Networks
  • NetASQ
  • Nortel
  • nProbe
  • Open vSwitch
  • Plixer
  • Solera
  • Saisei Networks
  • SonicWall
  • VMware
  • Xirrus
  • YAF
  • ZTE

Top 5 Reasons to use NetFlow Traffic Analyzer

  1. Monitors network bandwidth
  2. Monitor traffic patterns
  3. Identifies which users, applications, & protocols are consuming the most bandwidth
  4. Highlight the IP addresses of endpoints
  5. Analyse Cisco® NetFlow, Juniper® J-Flow, IPFIX, sFlow® , Huawei NetStream™ & other flow data
  6. Quick deployment cycle

FlowSet ID

The FlowSet ID is used to distinguish template records from data records. A template record always has a FlowSet ID in the range of 0-255. Currently, the template record that describes flow fields has a FlowSet ID of zero and the template record that describes option fields (described below) has a FlowSet ID of 1. A data record always has a nonzero FlowSet ID greater than 255.

Length

Length refers to the total length of this FlowSet. Because an individual template FlowSet may contain multiple template IDs (as illustrated above), the length value should be used to determine the position of the next FlowSet record, which could be either a template or a data FlowSet.

Length is expressed in Type/Length/Value (TLV) format, meaning that the value includes the bytes used for the FlowSet ID and the length bytes themselves, as well as the combined lengths of all template records included in this FlowSet.

Template ID

As a router generates different template FlowSets to match the type of NetFlow data it will be exporting, each template is given a unique ID. This uniqueness is local to the router that generated the template ID.

Templates that define data record formats begin numbering at 256 since 0-255 are reserved for FlowSet IDs.

Field Count

This field gives the number of fields in this template record. Because a template FlowSet may contain multiple template records, this field allows the parser to determine the end of the current template record and the start of the next.

Field Type

This numeric value represents the type of the field. The possible values of the field type are vendor specific. Cisco supplied values are consistent across all platforms that support NetFlow Version 9.

At the time of the initial release of the NetFlow Version 9 code (and after any subsequent changes that could add new field-type definitions), Cisco provides a file that defines the known field types and their lengths.

Field Length

This number gives the length of the above-defined field, in bytes.

Additional resources & papers

Behind IPFIX and NetFlow there is a huge community, this community shares papers, researches and tools. In this section, you will find a list of usable IPFIX and Netflow sources.

  1. BotSuer: Suing Stealthy P2P Bots in Network Traffic through Netflow Analysis
  2. Malware Detection From The Network Perspective Using NetFlow Data
  3. Detecting Worms and Malware with NetFlow: Network Threat Detection
  4. Malicious Encrypted Traffic Detection HITCON CMT 2018
  5. Malware Traffic Detection using Tamper Resistant Features
  6. Peer-to-Peer Botnet Detection Using NetFlow
  7. Detecting malicious network activity using flow data and learning automata
  8. Malware Detection by Analysing Encrypted Network Traffic with Neural Networks
  9. Plugging Network Security Holes using NetFlow
  10. Recommendations for Network Traffic Analysis Using the NetFlow Protocol
  11. Malware Detection by HTTPS Traffic Analysis
  12. Filtering automated polling traffic in NetFlow data
  13. Encrypted Threat Analytics (ETA)
  14. Content Agnostic Malware Detection in Networks
  15. Epidemiology of Browser-Based Malware
  16. Distributed malware detection
  17. Detection of Malicious Network Behaviour in Encrypted Network Traffic
  18. Flow-based Compromise Detection
  19. Flow Monitoring Explained: From Packet Capture to Data
  20. Analyzing a personalized network system through netflow