Professional Documents
Culture Documents
IBM Corporation
jenny_totterdell@uk.ibm.com
Serviceability logs
Trace Logging
WebSEAL HTTP Trace Logging
Debugging Java Runtime Issues
GSKit Traces
Must Gather Information for Support
Capturing Core Files
System_status script
Question/Answer Session
Log Files
Installation Logs
If the easy installation programs are used, the log files are written to the temp directory
Windows - %TEMP% (e.g. C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp)
UNIX - typically /tmp or /var/tmp
Component
Policy server
msg__ammgr_install.log
msg__amproxy_install.log
Authorization server
msg__amacld_install.log
Runtime
msg__amrte_install.log
Java runtime
msg__amjrte_install.log
ADK
msg__amadk_install.log
msg__amwpm_install.log
WebSEAL
msg__amweb_install.log
msg__amwebadk_install.log
msg__amwpiismp_install.log
msg__amismp.log
msg__amars_install.log
msg__ampfs_install.log
msg__ldaps_install.log
Configuration Logs
Messages generated during the configuration process are stored
within Tivoli Access Manager configuration log files.
Component
Base
msg__config.log
msg__PDJrteCfg1.log
WebSEAL
msg__amweb_config.log
msg__pdwpicfg.log
Serviceability Logs
Examples of serviceability logs:
msg__pdmgrd_utf8.log
msg__webseald-default.log
Message Format
A message consists of:
Date
Message Number (unique 32-bit decimal or hexadecimal value)
Process Name
Priority (e.g. WARNING)
Component information (including file name)
Types of Messages
Notice (Notice_verbose)
Does not directly require action, such as information about running state
Warning
Results may not be as desired but the program continues to function
normally.
Error
The product continues to function, but some services or functionality might
not be available
Fatal
Unrecoverable error, the process encountering the error usually terminates
Message Examples:
Notices:
2005-08-09-09:07:31.814+00:00I----- 0x1354A0A0 pdmgrd NOTICE ivc general
ivmgrd.cpp 743 0x00000001 Server startup
Server startup message
Warning:
2003-10-31-23:09:45.457+00:00I----- 0x38CF0131 webseald WARNING wwa server listenssl.c 167 0x00000044 The 'ssl_writechunk' routine failed for 'gsk_secure_soc_write', errno
= 406
This error is common and normal for webseal and ssl, which is why they're reported as warnings.
Mainly they are due to network connectivity or the customer hitting the "stop" button on their browsers.
The reason you see several messages with the same timestamp is because the browsers tend to open
multiple simultaneous connections. Losing the network or hitting the "stop" button will cancel all
simultaneous connections.
406 is a GSKit return code GSK_ERROR_IO
Error:
2003-07-08-12:59:07.032+00:00I----- 0x1354A0B6 pdmgrd ERROR ivc general
LDAPClient.cpp 212 0x00000001 LDAP initialization failed: ira_rgy_init('tarsus', 636,
'cn=ivmgrd/master,cn=SecurityDaemons,secAuthority=Default', ***) = 113, 202
Connection to LDAP failed.
Fatal:
2004-12-09-14:42:32.391+01:00I----- 0x14C010A4 pdmgrd FATAL mgr general
e:\am510\src\ivmgrd\ivmgrd.cpp 252 0x00000ba4 HPDMG0164E The Policy Server could
not be started (0x14c01420).
Message ID Format
The message ID consists of 10 alphanumeric characters, where the sequence is
XXXYY####Z:
Subsystem
HPD
Base
DPW/HPW
WebSEAL
AWD
AWL
AWX
AMZ
Description
Informational message.
Warning message.
Error message.
Windows:
FATAL:STDERR:-;FILE:C:/PROGRA~1/Tivoli/POLICY~1/log/msg__fatal.log
ERROR:STDERR:-;FILE:C:/PROGRA~1/Tivoli/POLICY~1/log/msg__error.log
WebSEAL Logs
WebSEAL maintains three conventional HTTP log files that record
activity rather than messages:
request.log
logs HTTP requests, such as information on URLs that have been
requested and information on the client (e.g. IP address).
agent.log
records contents of the User_Agent: header in the HTTP request. Includes
data about the client browser, such as architecture or version number
referer.log
records the Referer: header of the HTTP request. Records the document
that contained the link to the requested document.
By default, these log files are located under the following directory:
UNIX: /var/pdweb/www/log/
Windows: C:\Program Files\Tivoli\PDWeb\www\log\
Request.log
Every response sent back by TAM is recorded with a one-line entry in
the request.log
Traces
Trace Logging
Unlike message logging, trace logging (or tracing) is not enabled by
default.
Examples:
Entries in /opt/PolicyDirector/etc/pdmgrd_routing (TAM 5.1)
Trace all components for the Policy Server at highest trace level
*:*.9:TEXTFILE.10.10000:/var/PolicyDirector/log/trace__%ld.log
Trace the Policy Server's LDAP client calls/LDAP Server return codes
ivc:ira.9:TEXTFILE.10.10000:/var/PolicyDirector/log/trace__
pdmgrd_ira.log
Examples:
pdadmin> server task webseald-instance trace set pdweb.debug 2
file path=/tmp/pdweb.debug.out
pdadmin> server task webseald-instance trace show
pdweb.debug 2
pdweb.snoop
Advantages:
Includes message bodies, responses from WebSEAL and client IP addresses
Decrypts HTTPS traffic
Disadvantages
Large trace files (4-5 chars per byte)
Messages are hex encoded (get ascii value for none control char)
Does not show WebSEAL user (unless iv_user header is sent to jnc)
packets do not correspond to network frames in network trace
Stopping traces
pdadmin>server task webseald-instance trace set
pdweb.debug 0
pdadmin>server task webseald-instance trace set
pdweb.snoop 0
Pdweb.debug Example
2005-08-09-14:04:57.878-05:00I----- thread(4) trace.pdweb.debug:2
/project/amweb510/build/amweb510/src/pdweb/wand/wand/log.c:309: ----------------- Browser ===> PD ----------------Thread_ID:13326
GET /test/ HTTP/1.1 Host: linux User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020903 Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/xmng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1 Accept-Language: en-us, en;q=0.50 Accept-Encoding: gzip,
deflate, compress;q=0.9 Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 Keep-Alive: 300 Connection: keep-alive
Authorization:*******************************
--------------------------------------------------2005-08-09-14:04:57.896-05:00I----- thread(4) trace.pdweb.debug:2
/project/amweb510/build/amweb510/src/pdweb/wand/wand/log.c:309: ----------------- PD ===> BackEnd ----------------Thread_ID:13326
GET / HTTP/1.1 via: HTTP/1.1 linux:443 user-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020903
iv_server_name: default-webseald-linux accept-charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 host: linux.net:8080 accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/xmng,image/png,image/jpeg,image/gif;q=0.2,text/css,*/*;q=0.1 keep-alive: 300 connection: close accept-language: en-us,
en;q=0.50 --------------------------------------------------2005-08-09-14:04:57.928-05:00I----- thread(4) trace.pdweb.debug:2
/project/amweb510/build/amweb510/src/pdweb/wand/wand/log.c:309: ----------------- PD <=== BackEnd ----------------Thread_ID:13326 HTTP/1.1 200 OK content-type: text/html last-modified: Wed, 06 Nov 2002 13:06:47 GMT date: Tue, 09 Aug
2005 19:04:57 GMT etag: "2137c-1254-3dc913e7" content-length: 4692 accept-ranges: bytes connection: close server:
IBM_HTTP_SERVER/1.3.26.2 Apache/1.3.26 (Unix) --------------------------------------------------2005-08-09-14:04:57.929-05:00I----- thread(4) trace.pdweb.debug:2
/project/amweb510/build/amweb510/src/pdweb/wand/wand/log.c:309: ----------------- Browser <=== PD ----------------Thread_ID:13326
HTTP/1.1 200 OK p3p: CP="NON CUR OTPi OUR NOR UNI" content-type: text/html last-modified: Wed, 06 Nov 2002 13:06:47 GMT
transfer-encoding: chunked date: Tue, 09 Aug 2005 19:04:57 GMT etag: "2137c-1254-3dc913e7" accept-ranges: bytes x-oldcontent-length: 4692 server: IBM_HTTP_SERVER/1.3.26.2 Apache/1.3.26 (Unix)
---------------------------------------------------
Java Issues
PDJrte Configuration
Verify the pdjrte has been configured properly
Created in <jre_home>
/PolicyDirector
/PolicyDirector/PD.properties contains key-value pairs used by the TAM java runtime
/PolicyDirector/PDJLog.properties contains key-value pairs used by Java Logging
/PolicyDirector/PDCA.ks CA certificate keystore. Used in subsequent calls to pdmgrd
(ie. SvrSslCfg)
Added in <jre_home>/lib/ext/
PD.jar admin and authorization java classes
ibmjcefw.jar java cryptography extension
ibmjsse.jar java secure sockets implementation
ibmjcaprovider.jar, US_export_policy.jar, local_policy.jar cryptography
ibmpkcs.jar, ibmpkcs11.jar public key cryptography standard support
jaas.jar java authentication and authorization service
US_export_policy.jar
local_policy.jar
msg__amj_notice.log
msg__amj_noticeverbose.log
GSKit Traces
GSkit Trace
To enable the trace, perform the following steps:
Specify the file in which the trace data is to be stored with the environment
variable GSK_TRACE_FILE. Reference the following example:
export GSK_TRACE_FILE=/tmp/mytracefile
Re-create the error.
The system will append a ".1" to the file name and then accumulate
about 25 megabytes of trace data. It will then close the
"/tmp/mytracefile.1" file, open a "/tmp/mytracefile.2" file which
accumulate 25 more megabytes of trace information. It will then close
that one, erase the first file, and start over.
The trace files are binary
System Data
core
daemon binary
libs.tar <--- this tar file includes all the libraries which the daemon loads.
README <- which contains the scenario when the core occurred
How to run
script must be run from the /opt/PolicyDirector/bin directory
must be plenty of space (at least 50-100mb) in the /tmp folder
create a README which contains the scenario for the core
Run senddata.pl
This file should be sent to IBM when the PMR is being opened
System_status Script
Sample of Information Gathered
O/S and patch levels
Resource and Environment data (Memory, disk space, environment
variables, locales, ulimits)
Network information (/etc/hosts, ip address, network devices, aliases)
TAM Configuration (configuration files, daemon build levels)
Questions