2.4. Detection

Irrespective of the protection and prevention mechanisms in place, it is possible that security attacks succeed and proceed in an organization’s network. It is extremely important to detect such attacks at the earliest onslaught so that action can be taken to stop further damage. More details of detection mechanisms and processes can be found in Bejtlich [5], Northcutt and Novak [12], and Amoroso [13].

Intrusion detection is the broad term used to describe the process for identifying the fact that a security attack has occurred (or is occurring). There is no single method for identifying attacks; typically, three methods are used. In host-based intrusion detection, audit trails, logs, deployment of suspicious code, logins, and so on are monitored to detect the occurrence of a security attack. In network-based intrusion detection, the packets entering the network are examined to see if they correspond to signatures of known security attacks. Anomaly-based intrusion detection looks for abnormal usage of network or system resources and flags potential problems.

Audit trail processing, used with host-based intrusion detection, is usually done offline. Care has to be taken to ensure that logs in hosts have not been tampered with. Logs from many hosts and systems may have to be correlated to detect attacks. Network-based intrusion detection is in real time as packets are captured. This can be problematic if the amount of data flowing into the network is extremely large, as the buffering capacity may be limited and packets may be dropped by an intrusion detection system (IDS). Using signatures of known attacks is a common technique used for intrusion detection. However, this may miss new and unidentified attacks. If signatures are made too specific, security attacks may be missed resulting in false negatives. If signatures are made too general, it is likely that some normal traffic and activity is flagged as a security attack resulting in false positives. Thus, careful tuning is often necessary to detect intrusions with low false positives or negatives. The algorithms used for intrusion detection can be fairly complex, making use of data mining, pattern matching, decision making, and so on.

Often, IDSs deploy sensors to probe or monitor the network or systems in question. It is necessary to deploy sensors on either side of a firewall to get an idea of the attacks that are being blocked. Multiple redundant sensors may be necessary depending on the network topology. Sensors themselves may have to be networked to correlate the collected data. Such a network may or may not be sepa- rate from the network that is being monitored. The Internet Engineering Task Force is working on formats for exchange of intrusion detection information.

It is possible that IDSs may themselves be subject to security attacks. There are techniques that Oscar may employ to thwart detection by IDSs (such as fragmentation, flooding, unrelated attacks). Recent trends in intrusion detection include distributed intrusion detection where system administrators from all over the world submit their monitored information to a service that then performs correlations to detect and identify attacks.

There are several kinds of intrusion detection systems available today including specialized appliances from vendors. SNORT is an open-source intrusion detection system that is available for free. While evaluating an IDS, it is necessary to consider the types of attacks that an IDS can detect, the operating systems it supports, whether it can handle huge amounts of traffic, if it is capable of displaying large amounts of data in an easily understandable manner, the management framework that it provides, and its complexity.

Today, combinations of IDSs and firewalls, called intrusion prevention systems (IPSs), are also available. Rate-based IPSs block traffic flows if they are seen to exceed normal rates. Signature-based IPSs block traffic when signatures of known security attacks are detected. Such systems are part of the intrusion response systems discussed in detail in Chapter 13.

Honeypots or Internet traps are systems used to detect and divert security attacks. Such systems look like real resources, perhaps with vulnerabilities. Their value lies in the fact that Oscar may probe them, launch attacks against them, and perhaps compromise some of the systems. Monitoring Oscar’s activities using honeypots can help detect other attacks against real systems or design methods of prevention.