Managing Network Security

Traffic Sentinel is not a replacement for existing perimiter security measures (firewalls, VPNs etc). Instead Traffic Sentinel works to complement perimiter security by providing visibility inside the firewall. Perimiter security alone is increasingly difficult to maintain as users enjoy greater mobility and internal resources are opened to business partners and customers. In addition, security can be compromised by inadvertent actions by employees (e.g. installing unauthorized software or hardware). Traffic Sentinel's broad visibility into activity throughout the network supports a number of security monitoring techniques:

Behavior Analysis identify suspicious activities (e.g. port scanning).
Policy Violations set network access policies and identify violations.
Unauthorized Devices involves identifying unauthorized changes to the network topology (e.g. wireless access points, new hosts, new/moved/removed equipment).
Audit Trail use the traffic history document network activity - even if server log files have been deleted.
Signature Analysis identify compromized hosts based on packet signatures.
Denial of Service identify the source(s) of denial of service attacks.

Often there is more than one way to apply a particular technique:

Signatures Packet signatures are applied as soon as flow data is received by Traffic Sentinel. If a security threat can be detected using a packet signature rule then this technique will provide the fastest notification of a problem. In addition, when a rule triggers, the packets that triggered the rule are captured and can be further analzed.
Reports Reports can run as often as once every 5 minutes and can generate alerts. Reports are often used to monitor for security violations and behaviors that are hard to express as simple packet signatures. Reporting can be used to detect historical security violations whereas packet signatures can only detect violations that occur after the rules are installed. This ability to go back into the past is very useful if you are reacting to problem you have recently identified; putting in place a new rule will help identify new violations, but the reporting capabilities will allow information about previously compromised hosts to be extracted.
Traffic The real-time traffic monitoring charts can be used to drill-down on specific types of traffic and identify suspicious behavior. This capability is useful for exploring and explaining unusual or suspicious traffic.

Which security monitoring techniques you choose to employ depends on the specific organizational requirements and vulerabilities.

Before proceeding, an important first step in configuring Traffic Sentinel for security monitoring is to divide the network into meaningful zones and groups. Providing a complete description of the address space and devices in the network allows Traffic Sentinel to identify local and non-local traffic, identify traffic between zones, locate hosts and to find unauthorized devices (see Configuring Traffic Sentinel).

Behavior Analysis

Network Behavior Anomaly Detection (NBAD) involves identifying unusual traffic patterns that are typically associated with worm activity, zero day attacks or other problems that may compromise network security.

A general technique for identifying anomalies is to compare the behavior of hosts in the network. Because problems tend to be rare and cause unusual behavior, affected hosts tend to show up as outliers when their traffic is compared to that of their peers ("Top N" analysis). This technique has a number of advantages over "baselining" approaches (baselining involves trending behavior over period of time and then highlighting changes in behavior). Comparing hosts to their peers does not involve the lengthy training period required to create a baseline. In addition this technique automatically adapts to normal, network-wide changes in traffic patterns.

Many of Traffic Sentinel's tools perform "Top N" analysis in order to highlight unusual behavior. In addition Traffic Sentinel maintains a history that can be used to create a baseline.

Scanning

Scanning of devices in the network is often associated with an attempt to find and compromise vulnerable hosts. Worms scan in order to propagate and a hacker will scan the network in order to identify security weaknesses.

Scanning behavior appears in the traffic as "fan" shape in which a single host sends traffic to a large number of destinations. The Traffic>Circles tool clearly shows the fan shape associated with scanning behavior.

In addition the Traffic>Trend tool can be used to look for scanning hosts by selecting one of the following Charts:

Top Sources by #Destinations will identify hosts that are scanning.
Top Destination Protocols by #Pairs will identify protocols that are being scanned.

While the Traffic tools are useful for visually exploring a security problem and identifying it's cause, automatic notification of scanning behavior ensures that attacks are not missed. There are two main approaches to automatic scan detection available in Traffic Sentinel:

Signatures
Reports

Signatures are used to match particular types of traffic. To use a signature to identify scanning you need to include a threshold in the rule. The following scanning rule will fire if scanning of TCP port 445 is detected (this port is associated with vulnerabilities in Windows).

alert tcp $HOME_NET any -> any 445 \
(\
  msg:"Microsoft-DS scan";\
  threshold: type scan, track by_src, count 5, seconds 600;\
  classtype:network-scan;\
  sid: 580002002;\
  rev:1;\
)

In this case the threshold is set to generate a notification if a host attempts to connect to 5 or more hosts in a 10 minutes (600 second) period. Use the Signatures>Configure menu to install signatures (see Help > Signatures).

Alternatively, a report can be used to detect scanning behavior. Go to the Reports>Run menu and access the "Security>Compromised Hosts>Port Scanning Activity" query.

Again we are looking for scanning on TCP:445. This report (if scheduled to run every 10 minutes) will create an event each time a host scans more than 5 addresses in a 10 minute period (Note an event will only be generated if the Threshold is set). In addition the report is set to keep a list of all violations in the last 24 hours and to only create events when new violations are detected.

Addtional filtering may be required to avoid false alarms. Filters can be added to the signature rule or in the Where clause of a report. For example, network management systems will perform network discovery sweeps and can trigger alerts. Creating a "Management" zone in the configuration allows this traffic to be excluded.

Unassigned Address Space

Attempting to connect to unused addresses is always suspicous. The best way to detect this behavior is to first create an UNASSIGNED zone in the Traffic Sentinel configuration (use the File>Configure menu) and to populate it with the unassigned address space in your network. The following rule (use the Signatures>Configure menu) can be used to detect and traffic to the UNASSIGNED zone:

alert ip any any -> ">>UNASSIGNED" any\
(\
  msg:"Activity to unused subnets";\
  classtype:attempted-recon;\
  sid:580001002;\
  rev:1;\
)

Alternatively, if you want to use a report, go to Reports>Run and access the queries under Traffic>General Queries. Any of these reports can be used to identify hosts sending traffic to the UNASSIGNED zone by simply including the following Where clause:

destinationzone = UNASSIGNED

The "Security>Access Policy>Inter-Group Access" query could be used to generate alerts when there is traffic to the UNASSIGNED zone (again by setting the Where clause). However, this query identifies the zone that traffic comes from, not individual address. To create a complete reporting solution, use Reports>Edit to create a new Security report and add both queries to the report (see Reporting tutorial for information on creating and scheduling reports). When scheduled this report will be able to generate events when traffic is seen to the UNASSIGNED zone and also identify the individual addresses sending this traffic.

Errors

Error messages are often generated when an attacker attempts to probe a host. The ICMP destination unreachable message (ICMP:3) is sent when a host attempts to connect to a service that is not available.

Whenever you are interested in profiling a particular host or protocol, the best starting points are the Search>Host or Search>Protocol menus. In this case enter "ICMP:3" (or "ICMP:*" if you don't know the port number) in the Search>Protocol form. Click on the Connections button in the result page to see a real-time chart and set the Chart to Top Destinations by #Sources.

The chart clearly shows that a single host (52-144.demo.inmon.com) is causing most of the "Destination Unreachable" traffic. Clicking on the host in the chart legend will provide more detail about the host (including its location).

This information is makes a useful report. Click on the Reports>Run menu and select the "Traffic>General Queries> Historical Top N Chart". Set the Category to "IP Destination", the Value to "Frames", the Units to "Frames" and set Where to:

destinationport = ICMP:3

See the Reporting tutorial for information on creating and scheduling this query in a report.

Policy Violations

Setting and enforcing acceptable usage policies is an important part of a proactive security mangement strategy. Effective policies reduce risk by limiting traffic to applications and hosts that perform necessary functions. Detecting and eliminating unauthorized devices and services reduces risk and improves performance by eliminating non-essential traffic. The network-wide surveillance capabilities of Traffic Sentinel provides an effective means of detecting policy violations.

Protect Critical Services

It is important to protect critical services that are required to make the network function. For example, it is easy for users to inadvertently install DNS and DHCP services in the network causing significant disruption. A hacker could use illicit DNS and DHCP servers to spy on confidential information.

The following signatures will detect unauthorized DNS and DHCP servers:

# Define Servers
var DNS_SERVERS = [10.1.5.1,10.1.6.1]
var DHCP_SERVERS = [10.1.5.254,10.1.6.254,10.1.7.254]

# Only authorized DNS servers allowed
alert udp !$DNS_SERVERS 53 -> $HOME_NET any \
(\
  msg:"Unauthorized DNS server";\
  classtype:policy-violation;\
  sid:580003002;\
  rev:1;\
)

# Only authorized DHCP servers allowed
alert udp !$DHCP_SERVERS 67 -> $HOME_NET any \
(\
  msg:"Unauthorized DHCP server";\
  classtype:policy-violation;\
  sid:580003003;\
  rev:1;\
)

As in previous examples this traffic can also be identified using Traffic Sentinel's reporting capability. Click on the Reports>Run menu and select either of the "Traffic>General Queries>Historical/Recent Traffic" reports. Enter the following Where clause to identify unauthorized DNS servers:

sourceport = UDP:53 
& sourceaddress != 10.1.5.1,10.1.6.1 
& destinationzone != EXTERNAL

Similarly, the following Where clause identifies unauthorized DHCP servers:

sourceport = UDP:67
& sourceaddress != 10.1.5.254,10.1.6.254,10.1.7.254
& destinationzone != EXTERNAL

Note You could also use the "Security>Compromised Hosts>Trojan Activity" report to generate alerts if unauthorized servers are detected. The report will automatically suppress duplicate events and maintain a list of violations. See the Reporting tutorial for information on creating and scheduling this query in a report.

Eliminate High Risk Activity

Setting and enforcing policies that eliminate non-essential traffic helps avoid future security problems. Each networked application requires management resources to ensure that it is properly configured and patched. Reducing the number of application allows network managers to focus on securing a smaller number of applications and reduces the chances that an unpatched host will become compromised.

Peer to peer applications (and their traffic) on the network can carry significant risks. Many peer to peer applications are of dubious origin and may contain backdoors, security holes or spyware. Users may inadvertently share proprietary information or download malware. In addition there is the legal risk of lawsuits from copyright holders.

The "Services>Peer to Peer Traffic>Top P2P Hosts" query can help identify hosts running peer to peer protocols.

Most peer to peer protocols generate UDP traffic among the peers. This query identifies hosts that communicate with a large number of peers using UDP. Certain common UDP protocols are excluded from the analysis (NTP, SNMP and DNS) since they would cause false positivies.

Protocols that can be used to diagnose network problems or manage network resources (e.g. traceroute and SNMP) can provide a hacker with powerful tools for network reconnaissance. Scouting the network is the first step to identifying potentially vulnerable systems. Detecting these early reconnaissance attempts helps eliminate this threat before the network is more seriously compromised.

The following signature can be used to identify hosts that are performing traceroute tests:

alert ip !$NETWORK_MANAGERS any -> $HOME_NET any \
(\
  msg:"Unauthorized traceroute";\
  classtype:policy-violation;\
  ttl:<3;\
  sid:580003005;\
  rev:1;\
)

Alternatively, you could use a report with the following Where clause to detect traceroute activity:

ipttl = 1,2,3
& sourcegroup != NetMgmt
& destinationgroup != EXTERNAL

Note This filter assumes you have defined a NetMgmt group in the configuration file that contains the addresses used by network managers.

The following signature can be used to identify hosts that are generating SNMP requests:

alert udp !$NETWORK_MANAGERS any -> $HOME_NET 161 \
(\
  msg:"Unauthorized SNMP request";\
  classtype:policy-violation;\
  sid:580003006;\
  rev:1\
)

Alternatively, you could use a report with the following Where clause to detect SNMP activity:

destinationport = UDP:161
& sourcegroup != NetMgmt
& destinationgroup != EXTERNAL

Enforce Access Rules

The "Security>Access Policy>Inter-Group Access" query is designed to detect traffic that violates access policies.

A policy may restrict the use of a protocol to specific hosts, groups or zones. When violations are detected they can generate notifications.

Unauthorized Devices

It is important to keep track of the devices attached to the network. The ability to detect and locate unauthorized equipment helps protect against security breaches.

The "Security>Recently Added/Moved Hosts" queries detect new addresses that appear on the network and tracks hosts if they are reconnected in a different location.

The "Security>Unauthorized Routers>Detect NAT Routers" can identify unauthorized routers on the network. NAT routers are particularly troublesome (especially if they also provide wireless access) since they authenticate as a single hosts but may provide uncontrolled network access to a large number of hosts.

Audit Trail

The traffic database that Traffic Sentinel constructs can act as a useful audit trail. An attacker will typically delete access logs on a compromised host, but much of their activity will have been recorded by Traffic Sentinel and can easily be accessed through the reporting tools.

For example, suppose that host 10.1.5.22 was compromised. Go to the Reports>Run page and select the "Traffic>General Queries>Historical Traffic" query. Set Select Keys to:

sourceaddress,sourceport,destinationaddress,destinationport

and use he following Where clause to identify any traffic to or from this host.

ipsource = 10.1.5.22 | ipdestination = 10.1.5.22

Signature Analysis

Certain vulnerabilities or attacks are best described using packet signatures. Signatures can be highly specific, looking for fields in the packet header and patterns in the payload.

For example, the following signature detects traffic associated with the NACHI/Welchia worm:

alert icmp $HOME_NET any -> any any \
(\
  msg: "NACHI/Welchia";\
  content: "|aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa|";\
  dsize:64;\
  itype: 8;\
  icode: 0;\
  classtype:trojan-activity;\
  sid: 10000008;\
  rev: 1;\
)

Traffic Sentinel uses a subset of the Snort syntax to describe packet signatures (see Help>Signatures>Configure). Snort signatures are commonly published on security web sites when a new threat is identified.

Denial of Service

Setting thresholds on frame rates and utilization will generally detect denial of service attacks. Traffic Sentinel will generate an alert when a threshold is exceeded and provide links to drill-down to the source of the traffic.

Support