Given a scenario, use appropriate network monitoring resources to analyze traffic. This chapter describes network monitoring tools, including various SNMP versions, syslog, and network traffic analyzers/sniffers. You configure SNMP in our CompTIA Network+ course.
SNMP
SNMP was first introduced in Chapter 5, and it will be covered in more detail in this chapter. SNMP is an Application Layer (Layer 7) protocol (using UDP ports 161 and 162) that facilitates the exchange of management information between network devices. An SNMP-managed network consists of a management system, agents, and managed devices. The management system executes monitoring applications and controls managed devices. It also executes most of the management processes and provides the bulk of memory resources used for network management. A network might be managed by one or more management systems.
An SNMP agent resides on each managed device and translates local management information data, such as performance information or event and error information caught in software traps, into a readable form for the management system. SNMP agents use GET requests that transport data to the network management software. SNMP agents capture data from Management Information Bases (MIBs), which are device parameter and network data repositories, or from error or change traps.
A managed element, such as a router, a switch, a computer, or a firewall, is accessed via the SNMP agent. Managed devices collect and store management information, making it available through SNMP to other management systems with the same protocol compatibility. Figure 24.1 below illustrates the interaction of the three primary components of an SNMP-managed network:
Figure 24.1 – SNMP Network Component Interaction
Referencing Figure 24.1 above, the managed router is the SNMP-managed device. Logically residing on the device is the SNMP agent. The SNMP agent translates local management information data, stored in the management database of the managed device, into a readable form for the management system, which is also referred to as the Network Management Station (NMS).
When using SNMP, managed devices are monitored and controlled using three common SNMP commands: read, write, and trap. The read command is used by an NMS to monitor managed devices. This is performed by the NMS examining different variables that are maintained by managed devices. The write command is used by an NMS to control managed devices. Using this command, the NMS can change the values of variables stored within managed devices. Finally, the SNMP trap command is used by managed devices to report events to the NMS. Devices can be configured to send SNMP traps or informs to an NMS.
SNMP traps are simply messages that alert the SNMP manager of a condition on the network. An example of an SNMP trap could include an interface transitioning from an up state to a down state. The primary issue with SNMP traps is that they are unacknowledged. This means that the sending device is incapable of determining whether the trap was received by the NMS.
SNMP informs are SNMP traps that include a confirmation of receipt from the SNMP manager. These messages can be used to indicate failed authentication attempts or the loss of a connection to a neighbor router, for example. If the manager does not receive an inform request, then it does not send a response. If the sender never receives a response, then the inform request can be sent again. Thus, informs are more likely to reach their intended destination.
While informs are more reliable than traps, the downside is that they consume more resources on both the router and in the network. Unlike a trap, which is discarded as soon as it is sent, an inform request must be held in memory until a response is received or the request times out. In addition, traps are sent only once, while an inform may be resent several times if a response is not received from the NMS.
Figure 24.2 below illustrates the communication between the SNMP manager and the SNMP agent for sending traps and informs:
Figure 24.2 – SNMP Communication
The three versions of SNMP are versions 1, 2, and 3. Version 1, or SNMPv1, is the initial implementation of the SNMP protocol. SNMPv1 operates over protocols such as User Datagram Protocol (UDP), Internet Protocol (IP), and the OSI Connectionless Network Service (CLNS). SNMPv1 is widely used and is the de facto network-management protocol used within the Internet community.
SNMPv2 revises SNMPv1 and includes improvements in the areas of performance, security, confidentiality, and manager-to-manager communications. SNMPv2 also defines two new operations:
- GetBulk
- Inform
The GetBulk operation is used to retrieve large blocks of data efficiently. The Inform operation allows one NMS to send trap information to another NMS and then to receive a response. In SNMPv2, if the agent responding to GetBulk operations cannot provide values for all the variables in a list, then it provides partial results.
SNMPv3 provides the following three additional security services that are not available in previous versions of SNMP:
- Message integrity
- Authentication
- Encryption
SNMPv3 uses message integrity to ensure that a packet has not been tampered with in-transit. SNMPv3 also utilizes authentication, which is used to determine whether the message is from a valid source. Finally, SNMPv3 provides encryption, which is used to scramble the contents of a packet to prevent it from being seen by unauthorized sources.
Next, we will provide a few examples of the parameters monitored by an NMS (graphical view) via the SNMP protocol. Figure 24.3 below illustrates a sample report for interface utilization on a network device:
Figure 24.3 – SNMP Interface Utilization Example
Figure 24.4 below illustrates a sample report for CPU parameters (e.g., utilization, temperature, and fan) on a network device:
Figure 24.4 – SNMP CPU Monitoring Example
MIB browsers are dedicated software tools that manage SNMP-enabled network devices. They allow administrators to load standard and proprietary MIBs and issue SNMP requests to retrieve data or make changes to an agent. A sample interface example of an MIB browser is presented in Figure 24.5 below:
Figure 24.5 – MIB Browser Example
SNMP Configuration Example (Cisco IOS)
In this section, we will analyze the way SNMP is configured on a common network device, a Cisco router in this case. In Cisco IOS software, the snmp-server host [hostname | address] command is used to specify the hostname or IP address of the NMS to which the local device will send traps or informs. To allow the NMS to poll the local device, SNMPv1 and SNMPv2c require that a community string be specified for either read-only or read-write access using the snmp-server community <name> [ro | rw] global configuration command.
SNMPv3 does not use the same community-based form of security but instead uses user and group security. The following configuration example illustrates how to configure the local device with two community strings, one for read-only access and the other for read-write access. In addition, the local device is also configured to send SNMP traps for syslog to 1.1.1.1 using the read-only community string:
| Router(config)#conf t
Enter configuration commands, one per line. End with CNTL/Z. Router(config)#snmp-server community read_pw RO Router(config)#snmp-server community write_pw RW Router(config)#snmp-server host 10.10.10.1 traps password syslog |
Syslog
A syslog server (daemon) is an entity that listens to the logging messages that are sent to it. You cannot configure a syslog daemon to ask a specific device to send it syslog messages. In other words, if a specific device has no ability to generate syslog messages, then a syslog daemon cannot do anything about it. In the real world, corporations typically use dedicated software for syslog capturing. Additionally, freeware (such as the Kiwi Syslog daemon) is also available for syslog capturing.
Syslog uses User Datagram Protocol (UDP) as the underlying transport mechanism, so the data packets are unsequenced and unacknowledged. While UDP does not have the overhead included in TCP, this means that on a heavily used network, some packets may be dropped and therefore logging information will be lost. A syslog solution is comprised of two main elements:
- Syslog server
- Syslog client
The syslog client sends syslog messages to the syslog sever using UDP as the Transport Layer protocol, specifying a destination port of 514. These messages cannot exceed 1024 bytes in size; however, there is no minimum length. All syslog messages contain three distinct parts: the priority, the header, and the message.
The priority of a syslog message represents both the facility and the severity of the message in an 8-bit number. The first 3 least significant bits usually represent the severity of the message (with 3 bits, you can represent eight different severity levels) and the other 5 bits represent the facility. You can use these values to apply filters on the events in the syslog daemon.
| Note: The values mentioned above are generated by the applications on which the event is generated, not by the syslog server itself. |
Some of the most important benefits of using a dedicated syslog server for log collecting include:
- Centralized information
- Consolidated information
- Correlates logs between different devices
- Reporting and analytic options (text and graphics)
Figure 24.6 shows a syslog server sample output, after collecting a bunch of log messages with different priorities:
Figure 24.6 – Syslog Server Output Example
Syslog Configuration Example (Cisco IOS)
The logging priority levels sent by Cisco IOS devices are listed and described in Table 24.1 below:
Table 24.1 – Logging Priorities
| Level | Level Name | Syslog Definition | Description |
| 0 | Emergencies | LOG_EMERG | This level is used for the most severe error conditions, which render the system unusable. |
| 1 | Alerts | LOG_ALERT | This level is used to indicate conditions that need immediate attention from administrators. |
| 2 | Critical | LOG_CRIT | This level is used to indicate critical conditions, which are less than Alerts but still require administrator intervention. |
| 3 | Errors | LOG_ERR | This level is used to indicate errors within the system; however, these errors do not render the system unusable. |
| 4 | Warnings | LOG_WARNING | This level is used to indicate warning conditions about system operations that did not complete successfully. |
| 5 | Notifications | LOG_NOTICE | This level is used to indicate state changes within the system (e.g., a routing protocol adjacency transitioning to a down state). |
| 6 | Informational | LOG_INFO | This level is used to indicate informational messages about the normal operation of the system. |
| 7 | Debugging | LOG_DEBUG | This level is used to indicate real-time (debugging) information that is typically used for troubleshooting purposes. |
In syslog, the facility is used to represent the source that generated the message. This source can be a process on the local device, an application, or even an operating system. Facilities are represented by numbers (integers). In Cisco IOS software, there are eight local use facilities that can be used by processes and applications (as well as the device itself) for sending syslog messages. By default, Cisco IOS devices use facility local7 to send syslog messages. However, it should be noted that most Cisco devices provide options to change the default facility level. In Cisco IOS software, the logging facility [facility] global configuration command can be used to specify the syslog facility. The options available with this command are as follows:
| R1(config)#logging facility ?
auth Authorization system cron Cron/at facility daemon System daemons kern Kernel local0 Local use local1 Local use local2 Local use local3 Local use local4 Local use local5 Local use local6 Local use local7 Local use lpr Line printer system mail Mail system news USENET news sys10 System use sys11 System use sys12 System use sys13 System use sys14 System use sys9 System use syslog Syslog itself user User process uucp Unix-to-Unix copy system |
To send messages via syslog, you must perform the following sequence of steps on the device:
- Globally enable logging on the router or switch using the logging on configuration command. By default, in Cisco IOS software, logging is enabled; however, it is only enabled to send messages to the console. The logging on command is a mandatory requirement when sending messages to any destination other than the console.
- Specify the severity of messages to send to the syslog server using the logging trap [severity] global configuration command. You can specify the severity numerically or using the equivalent severity name.
- Specify one or more syslog server destinations using the logging [address] or logging host [address] global configuration commands.
- Optionally, specify the source IP address used in syslog messages using the logging source-interface [name] command. This is a common practice on devices with multiple interfaces configured. If this command is not specified, then the syslog message will contain the IP address of the router or switch interface used to reach the server. If there are multiple interfaces for redundancy, this address may change when the primary path (interface) is down. Therefore, it is typically set to a Loopback interface.
The following configuration example illustrates how to send all informational (level 6) and below messages to a syslog server with the IP address 192.168.1.254:
| Router(config)#logging on
Router(config)#logging trap informational Router(config)#logging 192.168.1.254 |
This configuration can be validated using the show logging command, as illustrated below:
| Router#show logging
Syslog logging: enabled (11 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled) Console logging: disabled Monitor logging: level debugging, 0 messages logged, xml disabled, filtering disabled Buffer logging: disabled, xml disabled, filtering disabled Logging Exception size (4096 bytes) Count and timestamp logging messages: disabled No active filter modules. Trap logging: level informational, 33 message lines logged Logging to 192.168.1.254(global) (udp port 514, audit disabled, link up), 2 message lines logged, xml disabled, filtering disabled |
When configuring logging in general, it is important to ensure that the router or switch clocks reflect the actual current time, which allows you to correlate the fault data. Inaccurate or incorrect timestamps on log messages make the fault and problem isolation using a filtration and correlation process very difficult and very time consuming. In Cisco IOS devices, the system clock can be configured manually or the device can be configured to synchronize its clock with a Network Time Protocol (NTP) server.
Traffic Analysis
The packets that traverse the network carry valuable data that can be used to obtain a lot of information which can be used in decision making. An important parameter that can be monitored is bandwidth utilization and this can include the following associated information:
- Which applications are using the bandwidth
- Which users are generating most of the traffic
- What the peak hours are
Knowing all these details, you can make the right decisions and take proactive measures, including increasing the bandwidth on the network. Traffic analysis also provides information regarding application response times and this can help ensure that the business’s critical applications are performing as expected.
By gathering traffic statistics over a period of time, you can create trends and predict what will happen to the network in the future based on past performance. There are three ways you can perform traffic analysis:
- Gather the packets and perform statistics
- Query the network devices the traffic passes through for network statistics
- Use standalone probes at different points in the network
Dedicated packet analyzers are placed in the traffic path to read every packet that passes through the link to create statistics. These appliances can store either the full data or just the relevant information for statistical purposes (e.g., source, destination, protocol, size, etc.).
Some devices have built-in capabilities to gather traffic statistics using the following protocols:
- SNMP
- RMON
- NetFlow
Once you have the data you can generate a lot of interesting statistics and reports, including:
- Bandwidth utilization reports
- Errors statistics
- Application response times
- TCP/UDP port usage
RMON
Remote network monitoring (RMON) is actually a management information base that allows you to proactively monitor LAN traffic in the network environment. It tracks individual data packets, the number and size of those packets, Broadcast packet tracking, network utilization, errors, and statistics.
RMON agents run on various network devices, such as routers, switches, and servers. If you don’t want to run RMON on those specific devices because of the overhead, you can configure special RMON workstations as probes on specific network segments. RMON offers the possibility to diagnose faults within the LAN and allows network tuning and planning for growth and utilization.
RMON is implemented in two versions: RMON 1 and RMON 2. RMON 1 operates only at the Physical and Data Link Layers, so it must only be used to probe, tune, plan, and look for faults on hubs (at the Physical Layer) and switches (at Layer 2). RMON 2 is the most recent version and it provides much more functionality, as it can be used for Network Layer applications (Layer 3) as well as Layers 4 through 7. Considering this, RMON 2 can monitor database servers, exchange servers, e-mail, and Web traffic.
| Note: RMON is documented in RFC 1757. |
NetFlow
A better solution to RMON in an enterprise-level environment is to use the NetFlow protocol. NetFlow is a monitoring and measurement technology superior to a simple SNMP/RMON solution in that it provides a lot more detail on the data that passes through a specific interface. NetFlow scales to a large number of interfaces and this makes it a great enterprise and service provider solution. NetFlow monitors IP flows on a device.
An IP flow is based on a set of five, and up to seven, IP packet attributes, which may include the following:
- Destination IP address
- Source IP address
- Source port
- Destination port
- Layer 3 protocol type
- Class of Service
- Router or switch interface
NetFlow is also a great solution for service providers because it supports customer service programs and uses popular data warehousing and data mining solutions that are critical for competitive vendor offerings, such as flexible accounting and billing based on application usage, the time of day, bandwidth utilization, and Quality of Service elements. NetFlow is also a great tool for network scalability planning and overall analysis, and it can help lower the organization’s TCO (total cost of ownership).
The NetFlow management architecture consists of three components, which are shown in Figure 24.7 below:
- NetFlow data export service
- NetFlow flow collector service
- NetFlow data analysis service
Figure 24.7 – NetFlow Management Architecture
The NetFlow data export service is at the top of the three-tier NetFlow architecture. This is where the data warehousing and data mining solutions occur and the accounting statistics for traffic on the networking devices are captured. It uses the UDP protocol to export data, which involves a three-part process:
- Data switching
- Data export
- Data aggregation
The data is then exported to the second tier – the NetFlow flow collector service. At this level, using servers and workstations, you can perform data collection, data filtering, aggregation, data storage, and file system management using the existing or third-party file systems.
Network data analysis can be found at the lowest tier (the Access Layer). At this level, you can use network planning tools, overall network analysis tools, and accounting and billing tools, as well as export data to various database systems or Excel spreadsheets.
NetFlow can be used for network traffic accounting, usage-based network billing, network planning, security, Denial of Service (DoS) monitoring capabilities, and network monitoring, in addition to providing information about network users and applications, peak usage times, and traffic routing. All of this makes it a very powerful maintenance, monitoring, and troubleshooting tool.
NetFlow Configuration Example
Three primary steps are required when configuring NetFlow (on a Cisco IOS platform), as follows:
- Configure the interface to capture flows into the NetFlow cache using the ip flow ingress interface configuration command on all interfaces for which you want information to be captured and stored in the flow cache. It is important to remember that NetFlow is configured on a per-interface basis only.
The NetFlow information is then stored on the local router and can be viewed using the show ip cache flow command on the local device. In the event that you want to export data to the NetFlow flow collector, two additional tasks will be required, as follows:
- Configure the Cisco IOS NetFlow version or format that you want to use via the ip flow-export version [1 | 5 | 9] global configuration command. NetFlow version 1 (v1) is the original format supported in the initial NetFlow releases. This version should be used when it is the only NetFlow data export format version that is supported by the application that you are using to analyze the exported NetFlow data. Version 5 exports more fields than version 1 does and is the most widely deployed version. Version 9 is the latest Cisco IOS NetFlow version and is the basis of a new IETF standard. Version 9 is a flexible export format version.
- Configure and specify the IP address of the NetFlow flow collector, and then specify the UDP port that the NetFlow flow collector will use to receive the UDP export from the Cisco device using the ip flow-export destination [hostname | address] <port> [udp] global configuration command. The [udp] keyword is optional and does not need to be specified when using this command because UDP is the default transport protocol used when sending data to the NetFlow flow collector.
The following output illustrates how to enable NetFlow for a specified router interface:
| Router#config t
Enter configuration commands, one per line. End with CNTL/Z. Router(config)#interface Serial0/0 Router(config-if)#ip flow ingress Router(config-if)#end |
Following this configuration, the show ip cache flow command can be used to view the statistics collected in the flow cache, as illustrated in the output below:
| Router#show ip cache flow
IP packet size distribution (721 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .980 .016 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .002 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 278544 bytes 4 active, 4092 inactive, 56 added 1195 ager polls, 0 flow alloc failures Active flows timeout in 30 minutes Inactive flows timeout in 15 seconds IP Sub Flow Cache, 21640 bytes 4 active, 1020 inactive, 56 added, 56 added to flow 0 alloc failures, 0 force free 1 chunk, 1 chunk added last clearing of statistics never Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) ——– Flows /Sec /Flow /Pkt /Sec /Flow /Flow TCP-Telnet 2 0.0 34 40 0.0 10.5 15.7 TCP-WWW 2 0.0 9 93 0.0 0.1 1.5 UDP-NTP 1 0.0 1 76 0.0 0.0 15.4 UDP-other 42 0.0 5 59 0.0 0.0 15.7 ICMP 5 0.0 10 64 0.0 0.0 15.1 Total: 52 0.0 7 58 0.0 0.4 15.1
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts Se0/0 150.1.1.254 Local 10.0.0.1 01 0000 0800 339 Se0/0 10.0.0.2 Local 1.1.1.1 06 C0B3 0017 7 Se0/0 10.0.0.2 Local 10.0.0.1 11 07AF D0F1 1 Se0/0 10.0.0.2 Local 10.0.0.1 11 8000 D0F1 10 Se0/0 150.1.1.254 Local 10.0.0.1 01 0000 0800 271 Se0/0 10.0.0.2 Local 1.1.1.1 06 C0B3 0017 59 |
The following output illustrates how to configure and enable NetFlow data collection for the specified router interfaces, and then export the data to a NetFlow flow collector with the IP address 150.1.1.254 over UDP port 5000 using NetFlow version 5 or the version 5 data format:
| Router(config)#interface Serial0/0
Router(config-if)#ip flow ingress Router(config-if)#exit Router(config)#interface FastEthernet0/0 Router(config-if)#ip flow ingress Router(config-if)#exit Router(config)#interface Serial0/1 Router(config-if)#exit Router(config)#ip flow-export version 5 Router(config)#ip flow-export destination 150.1.1.254 5000 Router(config)#exit |
Following this configuration, the information collected can then be viewed using an application reporting tool on the NetFlow flow collector. Despite the export of the data, the show ip cache flow command can still be used to view statistics on the local device, which can be a useful tool when troubleshooting network issues or problem reports.
Summary
SNMP is an Application Layer (Layer 7) protocol (using UDP ports 161 and 162) that facilitates the exchange of management information between network devices. An SNMP-managed network consists of a management system, agents, and managed devices. The management system executes monitoring applications and controls managed devices. It also executes most of the management processes and provides the bulk of memory resources used for network management. A network might be managed by one or more management systems.
An SNMP agent resides on each managed device and translates local management information data, such as performance information or event and error information caught in software traps, into a readable form for the management system. SNMP agents use get requests that transport data to the network management software. SNMP agents capture data from Management Information Bases (MIBs), which are device parameter and network data repositories, or from error or change traps.
A syslog server (daemon) is an entity that listens to the logging messages that are sent to it. You cannot configure a syslog daemon to ask a specific device to send it syslog messages. In other words, if a specific device has no ability to generate syslog messages, then a syslog daemon cannot do anything about it. In the real world, corporations typically use dedicated software for syslog capturing. Additionally, freeware (such as the Kiwi Syslog daemon) is also available for syslog capturing.
Syslog uses User Datagram Protocol (UDP) as the underlying transport mechanism, so the data packets are unsequenced and unacknowledged. While UDP does not have the overhead included in TCP, this means that on a heavily used network, some packets may be dropped and therefore logging information will be lost. A syslog solution is comprised of two main elements:
- Syslog server
- Syslog client
The packets that traverse the network carry valuable information that can be used to obtain a lot of information. An important parameter that can be monitored is bandwidth utilization and this can include the following associated information:
- Which applications are using the bandwidth
- Which users are generating most of the traffic
- What the peak hours are
Some devices have built-in capabilities to gather traffic statistics using the following protocols:
- SNMP
- RMON
- NetFlow
Configure SNMP in our 101 Labs – CompTIA Network+ book.
