AIX Operating System Hardening Procedures & Security Guide

By Michael Desrosiers m3ip Inc. Email: mdesrosiers@m3ipinc.com Web Site: http://secure-it-consulting.com

1.1 Preamble
IBM has positioned AIX 5 L version 5.1, as the new standard in Unix operating systems. It is built upon AIX 4.3.3 and provides improvements in critical areas such as reliability, availability, performance and security. The recommended way to harden the AIX Operating System is to use the principle of least privilege. If the user does not need the service, they are not allowed to access that service. Also if the server is to be an application server, only allow those specific services like ports 80 443 and 8080 to the server. There is a security principle that says you should configure computers to provide only selected network services. The basic idea is this; every network service you offer is an opportunity for the bad guys (alternatively a risk to your system). That's not to say that you shouldn't offer any services -- a web application server that doesn't offer web services isn't very useful. Instead, the principle says you should have a good understanding of network services and you should not offer any service unless there are very good reasons for doing so. This paper offers reasons to harden both server and network services for AIX 5.1 -- an application of the security principle. Some security packages address the problem by stripping all (or nearly all) network services and then instruct you to be careful about what you add to the system. That's a great approach but requires that you "get your hands on" the system before anyone layers anything onto it and you understand what you're adding to the system when you add it back in. These are two conditions that do not apply at many sites. The approach here is different. We will consider services offered by the AIX 5.1 operating system, try to explain what each does, note the risks involved with each and make recommendations about what one ought to do to mitigate the risk.

1.1.1 Security Planning and Framework

Planning This is the part of the plan where you must define the overall security policies and goals. In many Organizations, this initial step is performed at the corporate level, and is likely to have already been completed. How much security is needed? How much security can your business afford? What is the crown jewel that you are protecting?

Architecture phase.

This is where the design of your environment is defined to meet the requirements of the planning

What are the weakest points in your environment? What would be the nature of the attempted attacks? Where would the exploits come from? Internal? External? Where is your company focused? Border? Perimeter? Implementation This is where the infrastructure is built from the architectural design.

Start with securing the servers and working out towards the perimeter. Start with one security package and rollout to the other servers. Start from the top down, in other words, physical layer, network layer, etc.

Monitoring Once the infrastructure is built, you will need to continuously monitor it for vulnerabilities and suspected attacks. A better approach might be to schedule weekly audits, so as not to choke the network with useless snmp traffic. Problems that are found here should then be addressed through the previous phases in order to find the best resolution possible.

Application logs System logs (syslog, sulog, wtmp, lastlogin, failedlogin, etc.) Audit logs System errors (errlog) System performance (vmstat, iostat, ptx, sar, wlmstat, etc.) Network performance (no, netstat, netpmon, etc.)

Filesystems and permission structures File Integrity (tripwire, AIDE, md5, etc.)

nt Response This is the phase that you must address your worse fears. The worst time to begin working on this phase is after an attack or breach that has already occurred. e spent in the beginning considering how you should respond to a real attack will pay for itself many times over if you are ever in this situation. You must think of this Pree thinking.

Identify the severity of the breach. Start an outline or working document for evidence gathering. Work methodically from the inside to the outside of your environment. Start at physical layer and work your why through. Have a checklist to work off of before the event takes place. Document everything you do and validate it. If additional help is needed have a vendor contract in place.

Policy Considerations

ganization's security policy for networked systems should require that a detailed computer deployment plan be developed, implemented, and maintained whenever computers g deployed. Access to your deployment plan should only be given to those who require the information to perform their jobs. All new and updated servers be installed,

red, and tested in a stand-alone mode or within test networks (i.e., not connected to operational networks). You must present a policy that defines in detail appropriate r within its I/T infrastructure. All servers present a warning banner to all users indicating that they are legally accountable for their actions and, by using the servers; they are ing to having their actions logged.

ements

and Procedures

ust develop a server deployment plan that includes security issues. Most deployment plans address the cost of the computers, schedules to minimize work disruption, plications software, and user training. In addition, you need to include a discussion of security issues. You can eliminate many networked systems vulnerabilities and prevent oblems if you securely configure computers and networks before you deploy them. Vendors typically set computer defaults to maximize available functions, so you usually efaults to meet your organization's security requirements. You are more likely to make decisions about configuring computers appropriately and consistently when you use a signed deployment plan. Developing such a plan will support you in making some of the hard trade-off decisions between functionality and security. Consistency is a key , because it fosters predictable behavior. This will make it easier for you to maintain secure configurations and help you to identify security problems (which often manifest viations from common, expected behavior). Refer to the better practice that keeping the AIX operating system and applications software up to date is an essential part of this

Services Identification

the purpose of each computer. Document how the computer will be used.

er the following:

What categories of information will be stored on the computer? What kind of information will be processed on the computer? What are the security requirements for that information? What network service(s) will be provided by the computer?

What are the security requirements for those services?

y the network services that will be provided on the server. Servers as a general rule should be dedicated to a single service. This usually simplifies the configuration, which the likelihood of configuration errors. In the case of the servers, the application server should be limited to www or https services. The db2 server should be ports 50000 2inst1) and 50001 (db2idb2inst1). It also can eliminate unexpected and unsafe interactions among the services that present opportunities for intruders. In some cases, it may opriate to offer more than one service on a single host computer. For example, the server software from many vendors combines the file transfer protocol (FTP) and the xt transfer protocol (HTTP) services in a single package. It may be appropriate to provide access to public information via both protocols from the same server host but we do ommend this as it is a less secure configuration.

ne how the servers will be connected to your network. There are concerns relating to network connections that can affect the configuration and use of any one computer. rganizations use a broadcast technology such as Ethernet for their local area networks. In these cases, information traversing a network segment can be seen by any computer segment. This suggests that you should only place trusted computers on the same network segment, or else encrypts information before transmitting it. The servers should be own private subnet.

AIX Installation Procedures

p and follow a documented procedure for installing an operating system. I have compiled a separate document that pertains to this bullet. In this document, the steps to ent and install a base AIX 5.1 image are detailed and described with all the parameters that are set during installation. Make all your parameter choices explicit, even if they he default settings. (This may seem to be unnecessary, but it can prevent security problems if you subsequently reuse your scripts or configuration files to configure servers). xplicit choices will still be used even if the defaults have changed with new AIX releases. Your installation procedure should also specify the security-related updates or patches to be applied to the operating system. If possible, have a single person perform the installation procedure for each computer and capture each installation step in a documented (such as through using a checklist).

Authentication and Authorization

ost common approach is the use of passwords; but other mechanisms can be used, such as keys, tokens, and biometric devices (devices that recognize a person based on cal characteristics such as fingerprints or patterns of the retinal blood vessels). Because authentication mechanisms like passwords require information to be accessible to the cation software, carefully document how that information will be protected. Authentication data is critical security information that requires a high level of protection. You follow the security groups guidelines for administrative access into your sensitive data environment. In other words, password length of 8 characters with at least 2 alpha ers, etc. We will be discussing this in more detail in the recommendations section of this document.

mine how appropriate access to information resources will be enforced. For many resources, such as program and data files, the access controls provided by AIX are the most means to enforce access privileges. Also, consider using encryption technologies to protect the confidentiality of sensitive information. In some cases, protection isms will need to be augmented by policies that guide user's behavior related to their workstations. Identify the users or categories of users of the computer. The categories are n user roles that reflect their authorized activity. The roles are often based on similar work assignments and similar needs for access to particular information resources administrators, software developers, data entry personnel, etc. If appropriate, include groups of remote users and temporary or guest users. Document the categories of users l be allowed access to the provided services. You may need to categorize users by their organizational department, physical location, or job responsibilities. You also need a y of administrative users who will need access to administer the servers and possibly another category for backup operators.

to AIX servers should be restricted to only those administrators responsible for operating and maintaining the server. This will ensure that the server's users are d to those who are authorized to access the provided service and responsible for server administration. Determine the privileges that each category of user will have on the To document privileges, create a matrix that shows the users or user categories (defined in the previous step) cross-listed with the privileges they will possess. The privileges omarily placed in groups that define what system resources or services a user can read, write, change, execute, create, delete, install, remove, turn on, or turn off. Decide how

ill be authenticated and how authentication data will be protected. There are usually two kinds of authentication: (1) the kind provided with the operating system, commonly r authenticating administrative users and (2) the kind provided by the network service software, commonly used for authenticating users of the service. A particular software entation of a network service may use the provided authentication capability, and thus it may be necessary for users of that service to have a local identity (usually a local ) on the server.

Backup and Recovery

ent procedures for backup and recovery of information resources stored on the computer. Possessing recent, secure backup copies of information resources makes it possible to quickly restore the integrity and availability of information resources. Successful restoration depends on configuring the operating system, installing appropriate tools, and ng defined operating procedures. You need to document backup procedures including roles, responsibilities, and how the physical media that store the backup data are handled, and managed. Consider using encryption technologies like ssh to protect backups. Your backup procedures need to account for the possibility that backup files may have been mised by an undetected intrusion. Verify the integrity of all backup files prior to using them to recover systems.

nd Checklists

AIX 5.1 server tools

e the tools that are used in I/T environments today. These tools are freeware, but have been validated by there reliability over the last 5 10 years.

Tool
md5

Purpose
Validate integrity of file contents Verify integrity of directories and files on the server Log unauthorized connections to servers

Extent of usage
Daily (automated)

Comments
freeware

tripwire or AIDE

Daily (automated)

freeware

tcp_wrapper

Daily (Viewing of logs)

freeware

syslog

Collect log information for unauthorized entry on the server Log parsing tool, that makes log reader more bearable Monitors service/port connections to server To encrypt connections to servers Analyze packets on the servers interface Packet capturing tool Encapsulation/tunneling of Communication paths

Daily (Automated)

Part of Operating System

swatch

Daily (Automated)

freeware

lsof

Daily (Automated)

freeware

ssh

Daily (Automated)

freeware

tcpdump

Daily (Automated)

freeware

ethereal openssl

Daily (Automated)

freeware freeware

nmap

Network exploration tool and security scanner Network scanner and vulnerability assessment tool

Weekly(Automated)

freeware

nessus

Weekly (Automated)

freeware

st

AIX Security Checklist

3.2.1.1 AIX Environment Procedures

t way to approach this portion of the checklist is to do a comprehensive physical inventory of the servers. Serial numbers and physical location would be sufficient.

____

Record server serial numbers

____

Physical location of the servers

e want to gather a rather comprehensive list of both the AIX and pseries inventories. By running these next 4 scripts we can gather the information for analyze.