Monday, October 8, 2012

SIEM + scanner. Headache Pills?

Security systems are developed and adjusted to new threats all the time. The number of information resources, from which the data on the current security state is transferred, is getting bigger day by day. However, if you fail to detect and prevent threats timely, even hundreds of intrusion detection systems will be useless. And here the SIEM (Security Information and Event Management) systems come at help. These systems are in the focus of the article.

What is SIEM?

The SIEM system fulfills the following tasks:

  • Consolidation and storage of event logs from various resources — network devices, applications, OS logs, protection tools. Any information security standard provides technical requirements on event collection and analysis. They are needed not only to fulfill a standard requirement. There are situations when an incident is noticed too late, and events were erased long ago, or event logs are unavailable, and in fact it is impossible to find out the incident reasons. Moreover, connecting to each resource and event viewing will take too much time. Otherwise, without event analysis, there is a risk to learn about an incident in your company from the news.
  • Provision of tools for event analysis and incident investigation. Event formats differ in various resources. Text format in case of huge volumes is too tiresome; it reduces the possibility of incident detection. A part of products of the SIEM class unifies events and make them more readable, and the interface visualizes only important information events, focuses on them, allows filtering out not critical events.
  • Correlation and processing in accordance with rules. An incident cannot be judged by only one event. The simplest example is login failed — one event means nothing, but three and more such events with the same account can already indicate brute force attempts. Rules in SIEM, in the simplest case, are represented as RBR (Rule Based Reasoning) and contain a set of conditions, triggers, counters, an action script.
  • Automatic notification and incident management. The SIEM primary task is not only to collect events, but to automate the process of event detection and registration in its own log or an external system HelpDesk, and to inform about the event timely.

SIEM is able to detect:

Network attacks in internal and external perimeters
Virus epidemics or particular virus attacks, viruses not removed, backdoors and trojans
Unauthorized attempts to access confidential information
Errors and failures in information system functioning
Configuration errors in protection tools and information systems

The SIEM system is multipurpose due to its logic. However, for its tasks to be solved, useful correlation resources and rules are needed. Any information about an event (for instance, if a door of a particular room has opened) can be sent to the SIEM system and used.

Resources are selected on the basis of the following factors:

  • Severity of a system (value, risks) and information (processed and stored)
  • Validity and self-descriptiveness of event resources
  • Information channel coverage (not only external, but an internal network perimeter should be taken into account)
  • Solution of IT and IS tasks (ensuring continuity, incident investigation, policy compliance, information leakage prevention, etc.)

Primary SIEM resources

  • Access Control, Authentication — to control access to information systems and to use privileges
  • Event logs of servers and working stations — to control access, ensure continuity, comply with information security policies
  • Active network equipment (modification control and access, network traffic counters)
  • IDS\IPS. Notifications about network attacks, configuration changes, and device access
  • Antivirus protection. Notifications about software workability, databases, configuration and policy changes, malware
  • Vulnerability scanners. Inventory of assets, services, software, vulnerabilities, provision of inventory data and topological structure
  • GRC systems for recording of risks, threat severity, incident prioritization
  • Other systems of protection and compliance with IS policies (DLP, antifraud, device control, etc.)
  • Inventory and asset management systems — to control and detect new infrastructure assets
  • Netflow and traffic control systems

The SIEM solution usually consists of several components:

  • Agents installed on an information system under investigation (essential for operating systems; an agent is a resident program (service, demon), which collects event logs locally and transfers them to a server if possible).
  • Agents' collectors, which in fact are modules (libraries) for interpreting a particular event log or system.
  • Server collectors intended for prior event accumulation from various resources.
  • A server correlator responsible for collecting of information from collectors and agents and processing it in accordance with the rules and algorithms of correlation.
  • Database and storage server responsible for event log storing.

Event data is collected from resources with the help of the agents installed on them or remotely (using connections via NetBIOS, RPC, TFTP, FTP). The second option results in network and event resource loading, because some systems do not allow transmitting only those events, which have not been transferred yet, and transmit to SIEM the whole log weighing very often hundreds of megabytes. And it is not correct to remove a log each time when data is collected.

Events should not only be collected in a consolidated storage in case of an incident, but processed as well. Otherwise the solution will not justify your expenses. Of course, the SIEM toolset will save time needed for an incident investigation. However, SIEM is meant to detect and prevent threats timely. To fulfill this task, it is necessary to compose correlation rules taking into account company's relevant risks. These rules are not permanent and should be updated by experts all the time. Similar to intrusion detection systems, if a rule allowing detecting a typical threat is not created in a proper time, the attack is likely to be conducted. SIEM has an advantage over IDS — it is possible to specify general description of symptoms and use baseline statistics to monitor deviations from common behavior of information systems and traffic.

Its rules resemble the Snort rules vaguely. They describe threat criteria and reaction to them. I have explained the simplest example with login failed earlier. When applied, a more complicated example may be login failed in a particular information system with specification of a user group and remote object name. In case of fraud — distance parameters of two places where a bank card was last used within a small time interval (for instance, a client pays for petrol in Moscow and in 5 minutes somebody tries to withdraw 5,000 Euro in Australia).

Incident registration in its own or external HelpDesk system is not less important. First of all, incidents are documented. If an incident is registered, then there must be a person responsible for its elimination within a certain period of time. No incident will be missed (as it happens with notifications by email). Second, it provides incident statistics, which allows detecting problems (incidents of the same type, repeating incidents and incidents closed without elimination of an actual problem). Statistics and key indicators may be used to evaluate work efficiency of particular employees, IS departments, protection tools.

SIEM can help to make the threat detection process completely automated. If such a system is implemented correctly, an IS department reaches much higher level of service provision. SIEM allows paying attention only to very important threats, working not with events but with incidents, detecting abnormal behavior and risks, preventing financial losses.

It is important to realize that SIEM is not only the tool of information security but of the whole information technology. Strong correlation mechanisms can ensure continuity of IT service operation, detect outages of information and operating systems, hardware. Moreover, SIEM is a tool of automation. The most common example relevant for the majority of companies is the conflict of IP addresses. An easy RBR rule can notify about an incident long before a user call. Besides, the reasons can be eliminated with fewer costs, and therefore probable financial losses will be decreased.

Analyzing actual SIEM application, we have to accept that in the majority of cases the work of such systems is aimed at consolidation of logs from various resources. In fact, only SIEM hardware and software is used. If correlation rules have been already set, they are not updated.

SIEM and (or) vulnerability scanner

Due to some marketing publications, a myth that it is possible to protect the whole network perimeter with only one protection tool runs in the minds of integrators. You can often hear such questions as, "why do you require host IDS if it is only needed to use border ones?", "why do you want a security scanner if you have SIEM installed?", "what is the use of a vulnerability scanner, if IDS safely protects everything?" Let's check how things go in reality.

SIEM can correlate:

  • A known, described by correlation rules threat
  • A threat based on a general template
  • Abnormal behavior in case of deviation from a baseline
  • Deviation from a security policy based on the idea "everything what is not allowed is prohibited" (it is possible not in all SIEM systems)
  • Cause-effect relations if correlation algorithms such as CBR (codebook, smart), GBR (graph based), statistical, Bayesian are used

The last three algorithms are rarely applied in Russia. Not all SIEM systems can work with these correlation methods. Their use increases the cost of the system maintenance — a qualified specialist is needed to configure them, update, and maintain their workability. Of course, there are a lot of false notifications at the beginning of use. That is why companies very often just disable these detection mechanisms.
It turns out that only two simplest detection algorithms with a preset threat description are used. If a threat is new, it will not be detected. A vivid example is APT (Advanced Persistent Threat) in RSA, the developer of SIEM (uses its own system).

For the proper operation of these two algorithms, it is required to update data about threats all the time similar to IDS. As a result, threats are duplicated in IDS and SIEM (but SIEM correlation rules are updated much more rarely than IDS rules). Rules updating in the SIEM products is often missing — not all companies can afford a qualified analyst of SIEM and its rules, and besides this country lacks good specialists.

So in case of one-time configuration of correlation rules, an incident (for instance, a network attack) will be detected only if it is reported by another protection tool (IDS, for example).

Let's consider one more practical example. In case of a vulnerability, at first it can be revealed only by detecting particular criteria — software or plug-in version, various configuration parameters. You may know about a vulnerability beforehand, but you will learn the method of its exploitation detection only after the issue of a bulletin. Of course, a malware user can freely exploit this vulnerability during the whole period. All security tools will keep silent in the majority of cases, because they do not know the methods of detecting attacks exploiting this vulnerability. Even if updates and vulnerability elimination techniques have been issued, it is not always possible to fix it in a system (think of effort, testing necessity, incompatibility of different systems, sometimes it cannot be eliminated at all). Remaining risks are accumulated up to a dangerous state, but they can be and should be controlled.

Integration of a vulnerability scanner with SIEM allows combining several methods of threat detection and significantly increasing probability of timely detection. For instance, SIEM can detect an abnormal behavior through a baseline, but without information that an asset has a vulnerability SIEM will not be able to identify with what exactly this abnormal behavior is connected. If there is data from a vulnerability scanner, then SIEM will be able to conclude that this vulnerability is exploited.

With information about a vulnerability and asset severity from a vulnerability scanner, the SIEM system is able to prioritize incidents in accordance with their severity. First of all it will allow reacting to significant incidents important for business.

Vulnerability scanner is an excellent supplier of inventory information for SIEM (software versions, its configuration); this information can be used to detect an incident, to find out what it results from. For instance, the swap parameter of a file has been changed by a user company\p_kolya. The server is frequently restarted. It is quite typical, and only the SIEM system is able to reveal cause-effect relations. However, without integration with a vulnerability scanner, you will look for the reasons for quite a long time, and your company will endure financial losses because of the service delay.

Do you know all computers in your network? Do all of them belong to your company? Are all of them provided with security tools in accordance with the security policy? Are these security tools functioning and configured properly?

Use of an embedded into SIEM mechanism that checks compliance with internal policies and high-level standards without integration with a vulnerability scanner will not provide you with a complete picture, because very few technical requirements are used. A vulnerability scanner is intended not only to detect vulnerabilities, but to supply the major part of controls. You may not even once be notified about a virus, but it does not mean your antivirus protection functions correctly. You may not even have it. SIEM will never inform you that antivirus software is not installed or the option of file protection is disabled. And a vulnerability scanner can provide you not only with this but with other useful information as well.

No resource will be able to provide you with more detailed and complete information about a vulnerability in your system and possible ways of its exploitation (with regard to the network topological structure and configuration) than a vulnerability scanner. A vulnerability may be present in a system but remain unexploited (a network port is closed, a service is stopped, VLAN is organized on active network equipment, or firewall rules locked traffic to this port). Such information can reduce remaining risks, help to apply resources only to necessary protection tools and to rule out false incidents.

Configuration management process, so complicated to be implemented, becomes simple if SIEM and a vulnerability scanner are used together. You can analyze what has changed, who's made these changes and when, and you can also automatically evaluate what they've affected. You only need to compose correlation rules in SIEM and configure parameters of information forwarded from a scanner; the SIEM system will perform other logic itself.

Of course, a scanner without SIEM reduces risks and allows evaluating possible attack vectors as well. However, continuous asset scanning will increase their and network resource loading. A vulnerability can appear within an interval between scanning, and the bigger these intervals, the larger financial risks. SIEM keeps guarding your system within these intervals. The SIEM scripts allow forcing a vulnerability scanner to start running and information to be updated in case of any threat.

It is evident that the more effective information resources SIEM has, the more it is possible to detect a threat at the stage of its appearance. You can use SIEM and a vulnerability scanner separately. However, if they are used together, the risks with the hugest ROI index will be significantly minimized. This article has touched upon the simplest cases, which can be automated; when applied they are much more numerous.
Thank you for attention!

Author: Olesya Shelestova, Positive Research.


  1. I really think that availing some home or business security alarm systems would greatly help you to keep you, your family, your home and your business even if you are away. Apart from that it could also ensure your protection inside and outside. That's why I think Siem's development on this so called scanner will be really helpful.

  2. I would suggest that SIEM should file an llc to be able to assure that the company will be able t make it what ever legal or economical issues they have to face cause I'm telling you especially to those who choose to establish their business online this could serve as their security.

  3. Forming an incorporation induces a good effect on your company apart from that it could also help the company in promoting their businesses online and public awareness to the company's presence only implies that more likely there will be an increase in their sales market value.

  4. Awesome! It appears I've discovered another food! =) Keep up the considerable work and I'll continue perusing. Headache specialist

  5. Nice information about headache thanks for share this valuable info.