Pages

Monday, November 8, 2010

PCI DSS and Red Hat Enterprise Linux (Part #8)

Requirement 10: Track and monitor all access to network resources and cardholder data

Abstract

Technically implemented requirements given in this chapter refer to the syslog server, the kernel-level audit system auditd, NTP server settings, and an integrity control system. There are almost no analogous items in CIS standards.

10.2 Through interviews, examination of audit logs, and examination of audit log settings, perform the following.

The requirements stated in items 10.2.X refer to the auditd system, which should be installed (audit package) and started (chkconfig auditd on). Audit rules (which define what data is to be logged) should be added to the file /etc/audit/audit.rules; daemon settings are specified in /etc/audit/auditd.conf. For the both files, it is recommended to set access permissions chmod 600. By default, logs are stored in the directory /var/log/audit (however, the path can be redefined in auditd.conf), which is often located on a separate partition. Audit files are stored in text format, but it is better to use utilities aureport and ausearch to read them. Temporary audit rules (which are valid until daemon restart) can be defined using the auditctl command; while the rule syntax of this command is the same as the syntax of audit.rules, man auditctl is much more informative.

First of all, it is necessary to specify the rule of filtering needless messages like cwd (change working directory) in /etc/audit/audit.rules; this rule must be given in the first line:
-a exclude,always -F msgtype=CWD

To make the rules come into effect after the configuration file was changed, it is necessary to execute the following command:
service auditd reload

For correct operation of the audit system, the module pam_loginuid should be specified in the following string:
session required pam_loginuid.so

in files /etc/pam.d/{atd,crond,kdm,kdm-np,login,remote,sshd,xdm} (the list is valid for FC12). Moreover, you can deny users to log in when auditd is not running by using the parameter “require_auditd” of the module pam_loginuid.
It is also necessary to enable audit during OS boot-up by adding the following kernel option to /etc/grub.conf:
audit=1

10.2.2 Verify actions taken by any individual with root or administrative privileges are logged.

This rule can be added to /etc/audit/audit.rules in the following form:
-a always,exit -F euid=0 -F perm=wxa -k ROOT_ACTION

Here, the syntax of rules allows one to “permute the terms.” For example, the element “always,exit” can be put down as “exit,always”; “euid=0” can be put down as “euid=root”; in the substring “perm=wxa,” the block “wxa” can be put down as “axw,” etc. The syntax of rules is described in details in man auditctl.

In the given rule, it is specified that API calls (exit) made by users with EUID = 0 (including those who gained root privileges via su/sudo) and entailing command execution (x), data writing (w), or modification of file/directory attributes (a) are always logged. All such entries are marked with ROOT_ACTION label, which allows one to search for events and display them. From the example given below, one can see that the file /tmp/audit_me is at first created by the script (entries 1 and 2) and then it is opened in vi and overwritten (entries 3–14).

[root@rf12 audit]# ausearch -ts today -k ROOT_ACTION -f audit_me | aureport -i -f
File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 07/07/10 14:15:50 /tmp/audit_me open yes /bin/bash root 87709
2. 07/07/10 14:15:58 /tmp/audit_me rename yes /bin/sed root 87719
3. 07/07/10 14:51:51 /tmp/.audit_me.swpx open yes /bin/vi root 88098
[ … ]
13. 07/07/10 14:51:52 /tmp/audit_me chmod yes /bin/vi root 88108
14. 07/07/10 14:51:54 /tmp/.audit_me.swp unlink yes /bin/vi root 88109

10.2.3 Verify access to all audit trails is logged.

It is necessary to add certain entries for every file or directory (usually it is /var/log and all objects located in it) being under observation to the file /etc/audit/audit.rules. For these entries, two types of syntax are acceptable; the first one is:
-w /path/to/file_or_directory -p rwxa

Here, rwxa represents various flags (read/write/execute/access) (man auditctl).
The second syntax type is different for files and directories and looks as follows:
-a exit,always -S all -F dir=/path/to/dir
or
-a exit,always -S all -F path=/path/to/file

10.2.4 Verify invalid logical access attempts are logged.
10.2.5 Verify use of identification and authentication mechanisms is logged.

These messages are processed by daemon syslog, not auditd. By default, all system access events are registered in /var/log/secure, because /etc/syslog.conf is configured to add messages “authpriv.*” to this log file.
In FC12, rsyslog is installed instead of syslog. The corresponding configuration file is /etc/rsyslog.conf; it allows one to use the same syntax as in syslog.conf and specify advanced parameters (which are not considered in this article).

10.2.6 Verify initialization of audit logs is logged.

Configuring auditd is accordance with the requirement 10.2.3 implies logging of ALL system calls that affect log files, including those relating to deletion (API functions rmdir and unlink) and overwriting/nulling (truncate, open with flags O_RDWR, O_WRONLY, or O_TRUNC).
The second syntax type for this requirement (for deletion only, open() calls are not supported) can look as follows:

[root@rf12 log]# grep LOGS_INIT /etc/audit/audit.rules
-a exit,always -F dir=/var/log -S truncate -S unlink -S rename -S unlinkat -k LOGS_INIT
[root@rf12 log]# pwd
/var/log
[root@rf12 log]# rm -f boot.log-201005*
[root@rf12 log]# ausearch -ts today -k LOGS_INIT | aureport -i -f
File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 07/07/10 19:06:00 boot.log-20100517 unlinkat yes /bin/rm root 89799
2. 07/07/10 19:06:00 boot.log-20100511 unlinkat yes /bin/rm root 89798
3. 07/07/10 19:06:00 boot.log-20100524 unlinkat yes /bin/rm root 89800
4. 07/07/10 19:06:00 boot.log-20100531 unlinkat yes /bin/rm root 89801
[root@rf12 log]#

Note. When parameters of file operation commands are given in the form of relative paths, the utility aureport displays them as relative, too. To find out the absolute path, it is necessary to call for another command, not aureport (this command and details of its action are set off with bold below):

[root@rf12 log]# ausearch -ts today -k LOGS_INIT
time->Wed Jul 7 19:06:00 2010
type=PATH msg=audit(1278515160.180:89798): item=1 name="boot.log-20100511" inode=11777 dev=fd:01 mode=0100644 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:var_log_t:s0
type=PATH msg=audit(1278515160.180:89798): item=0 name="/var/log" inode=7923 dev=fd:01 mode=040775 ouid=0 ogid=501 rdev=00:00 obj=system_u:object_r:var_log_t:s0
type=SYSCALL msg=audit(1278515160.180:89798): arch=40000003 syscall=301 success=yes exit=0 a0=ffffff9c a1=bfa9a7b5 a2=0 a3=2 items=2 ppid=2604 pid=21257 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=112 comm="rm" exe="/bin/rm" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="LOGS_INIT"
[ … ]

In the last string, the AUID (UID received at login and registered by the module pam_loginud) is also given.

10.2.7 Verify creation and deletion of system level objects are logged.

It isn’t obvious what objects should be considered as system level objects. However, we can configure logging of events concerning files from directories /etc, /bin, /sbin, /usr/bin, /usr/sbin, /var/lib, /lib, /usr/lib, and /usr/libexec, and for 64-bit OSs – from directories /lib64 and /usr/lib64 additionally. It is recommended to configure not only the audit subsystem, but also an integrity control system (AIDE, which is described in 10.5.5, controls all system files by default).
The settings are similar to those given in 10.2.3 (the first syntax variant), for example:
-w /etc –p wa

This setting will result in appearance of the following entries in the audit log:

[root@rf12 audit]# touch /etc/auditme2
[root@rf12 audit]# rm -f /etc/auditme2
[root@rf12 audit]# ausearch -ts today -k ETC | aureport -i -f
File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 07/07/10 18:58:05 /etc/auditme2 open yes /bin/touch root 89795
2. 07/07/10 18:58:08 /etc/auditme2 unlinkat yes /bin/rm root 89796

10.3 Through interviews and observation, for each auditable event, perform the following:
10.3.1 Verify user identification is included in log entries.

The field auid (UID at login) is displayed in the output of aureport, the field euid (actual UID obtained in the result of execution of su/sudo/SUID/SGID) is displayed in the output of ausearch.

10.3.2 Verify type of event is included in log entries.
The command name and API calls also should be stored and displayed.

10.3.3 Verify date and time stamp is included in log entries.
They should be logged and displayed.

10.3.4 Verify success or failure indication is included in log entries.
In the output of ausearch, there is a string “success=”; ausearch allows one to search for events with a specified status (successful/failed: -sv yes | -sv no).

10.3.5 Verify origination of event is included in log entries.
It is given in the field “tty=” (the output of ausearch). /var/log/secure contains information about what time and from what source can a certain user enter the terminal.

10.3.6 Verify identity or name of affected data, system component, or resources is included in log entries.
It is also displayed as a result of ausearch execution.

10.4 Obtain and review the process for acquiring and distributing the correct time within the organization, as well as the time-related system-parameter settings for a sample of system components. Verify the following is included in the process and implemented:

10.4.а Verify that a known, stable version of NTP or similar technology, kept current per PCI DSS Requirements 6.1 and 6.2, is used for time synchronization.

To fulfill this requirement, it is necessary to install the package ntp (server part, which allows one not only to distribute time to the clients, but also to obtain it from superior hosts) or ntpdate (for clients).
The simplest way to synchronize time on the client side is to run ntpdate periodically (via cron) and specify remote synchronization servers to it. You can also run “ntpd –q”. On FC12, ntpdate may be started from /etc/init.d to synchronize time at OS startup, which is more suitable for clients than for servers.
To run NTPD daemon at system startup, execute the following command:
chkconfig ntpd on

Furthermore, it is necessary to set server parameters in /etc/ntp.conf (these settings will be described below).

10.4.b Verify that internal servers are not all receiving time signals from external sources. (Two or three central time servers within the organization receive external time signals, peer with each other to keep accurate time, and share the time with other internal servers.)

Here, it is necessary to configure internal company servers to receive time signals from external sources (except the situation when a company has its own source of precise time, which is important e.g. for mobile network operators and certification authorities), as well as to set synchronization permissions for clients. All these parameters are specified in the file /etc/ntp.conf.
Addresses of external servers are defined using the following command:
server a.b.c.d

Here, the server address may be represented both as domain name or IP address.
It is necessary to allow access for other network hosts so that they were able to receive precise time from the configured server; for this purpose, one should use the following directive:
restrict e.f.g.h mask w.x.y.z nomodify notrap

and specify the network address and mask.
After these operations are performed and ntpd is restarted, clients can check time synchronization using the following command:
ntpdate -b $NTPSERVER

Here, $NTPSERVER is the address of an internal NTP server.

Note. To ensure operation of company NTP servers in situations when connection to external NTP servers is lost or when the network is not connected to the Internet or other precise time sources at all, it is necessary to specify the following settings on the central NTP server:
server 127.127.1.0
fudge 127.127.1.0 stratum 10

After that, NTP server will take system clock values as values from a precise time source if other sources will be unavailable.

10.4.с Verify that specific external hosts are designated from which the timeservers will accept NTP time updates (to prevent a malicious individual from changing the clock). Optionally, those updates can be encrypted with a symmetric key, and access control lists can be created that specify the IP addresses of client machines that will be provided with the NTP service.

To define external hosts of precise time clearly, it is necessary to apply the settings described in the previous item. Furthermore, it is recommended to specify external sources in the form of IP addresses (instead of FQDN) to decrease the risk of attacks against DNS servers.
Access control lists (ACLs) are specified via the directive restrict as shown in item 10.4.b.

10.5.1 Verify that only individuals who have a job-related need can view audit trail files.

To fulfill this requirement, it is necessary to perform actions similar to those described in item 7.1.1.I. It is important to control not only access permissions of files and directories, but also their owners and groups, as well as ACLs.

10.5.2 Verify that current audit trail files are protected from unauthorized modifications via access control mechanisms, physical segregation, and/or network segregation.
This requirement is similar to item 10.5.1, but here, write permissions are controlled.

10.5.3 Verify that current audit trail files are promptly backed up to a centralized log server or media that is difficult to alter.

Event logs are maintained by two daemons, syslogd (rsyslogd) and auditd.
To make syslogd send event messages to an external server in real time, it is necessary to specify a string of the following type for every event type in the client file /etc/syslog.conf (or /etc/rsyslog.conf):
facility.priority @remote.host.address

To allow server to receive messages from network, it is necessary to use the option “r” (in the file /etc/sysconfig/syslog); for rsyslogd, it is necessary uncomment the following lines in the file /etc/rsyslog.conf:

$ModLoad imudp.so
$UDPServerRun 514

It is more difficult to configure remote transfer of auditd logs. For this purpose, it is necessary to install an additional package audispd-plugins and its dependencies:
yum install audispd-plugins

After that, it is necessary to edit the following configuration files to apply basic settings of log transfer on the client side (the host name is fc12):

/etc/audit/auditd.conf:
name_format = none
/etc/audisp/audispd.conf:
name_format = HOSTNAME

Otherwise, the host name will be duplicated in the course of remote sending. After that:

/etc/audisp/audisp-remote.conf:
remote_server = 10.111.113.27
port = 64667
/etc/audisp/plugins.d/au-remote.conf:
active = yes

It will be necessary to edit only one file on the server side (the host name is sles11, the address is 10.111.113.27). The port is chosen randomly; it is only required that the port was free and the client had the same settings.

/etc/audit/auditd.conf:
tcp_listen_port = 64667

After auditd is restarted, we can see the following information on client and on server:

[root@rf12 audit]# touch /etc/auditme.tmp
sles11:/var/log/audit # tail -f audit.log
[ … ]
node=rf12 type=SYSCALL msg=audit(1278592521.992:90379): arch=40000003 syscall=5 success=yes exit=3 a0=bfc647e8 a1=8941 a2=1b6 a3=ffffffff items=2 ppid=2604 pid=3475 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=112 comm="touch" exe="/bin/touch" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="ETC"
node=rf12 type=PATH msg=audit(1278592521.992:90379): item=0 name="/etc/" inode=32642 dev=fd:04 mode=040755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:etc_t:s0
node=rf12 type=PATH msg=audit(1278592521.992:90379): item=1 name="/etc/auditme.tmp" inode=5729 dev=fd:04 mode=0100644 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:etc_t:s0
[ … ]

Kerberos may be applied for client authorization and authentication.
Extended support of remote sending and processing of logs is described in [3].

10.5.5 Verify the use of file-integrity monitoring or change-detection software for logs by examining system settings and monitored files and results from monitoring activities.

The main system of integrity control for Linux is AIDE (aide package). The corresponding system configuration file is /etc/aide.conf; use man aide.conf to obtain additional information. It is supposed that AIDE will be started via cron (with option --check or --compare); the frequency of execution depends on host characteristics.
Standard settings of packaged AIDE include tracking of objects in /var/log. An important feature of these files is that they can be removed or renamed during system log authorization (the logrotate daemon). It is recommended to learn logrotate parameters (file /etc/logrotate.conf, directory/etc/logrotate.d/) before configuring AIDE so that it was not necessary to investigate a number of incidents every time after logrotate execution.

Attention. Calculation of checksums may take several minutes of full loading of one processor core. For example, calculation of checksums will take about 7 minutes on a virtual machine (VMWare, one Intel Core2, 3.06 GHz). The reasons are that firstly, AIDE doesn’t support multithreading, secondly, hashes are often calculated using not one, but several algorithms, and thirdly, the system is configured to verify all system objects (/etc, /boot, /bin, /usr, /sbin, /lib, /root). For example, we managed to reduce the calculating time one and a half times by specifying skip of directory /usr/share in the configuration file.

10.7.b Verify that audit logs are available for at least one year and processes are in place to restore at least the last three months’ logs for immediate analysis.

To fulfill this requirement, it is reasonable to set long-term data storage on the central log server (because item 10.5.5 requires to configure transfer of logs to an external host); here, it is recommended to use a DBMS instead of text files for data analysis. Examples of such installations can be found in [1] and [2]. Application of DBMS to store logs has the following principal advantages compared to the traditional approach: extended features of archiving and replication and web-oriented display supporting data filtering.

References:

2 comments:

  1. I am currently implementing the requirements for PCI DSS 10, and have found your article quite helpful!

    Also, OSSEC (opensource HIDS) is an excellent log analysis and incident response tool that can go hand in hand with some of the PCI requirements.

    ReplyDelete
  2. Danny,
    thank you for the feedback,
    we would make a review of third-party (i.e. not included in main distros and repos) open-source utilities in the future. In fact this soft may be really useful for high-level compliance standards.

    ReplyDelete