http://www.theiia.org/itaudit/index.cfm?

fuseaction=print&fid=380
Vol. 4, October 1, 2001

Audit Tools

Use of Computer-Assisted Audit Tools and Techniques

(CAATTs), Part 1
Readers' rating: 5 out of 5
By Charles Le Grand, CIA, CISA, CDP
CAATTs may be classified in the following groups:
Electronic Working Papers
Information Retrieval and Analysis
Fraud Detection
Network Security
Electronic Commerce and Internet Security
Continuous Monitoring
Audit Reporting
Database of Audit History
Computer Based Training
Time Tracking
As audit tools grow more powerful and sophisticated, they are also becoming easier to learn and use.
And, at the same time, they also must fit into a complex and ever changing environment. Features of
audit software can easily conflict with features of other software on the computer or network, and must be
carefully managed.
As tools become more powerful, auditors may use features or services provided in the software that
command considerable system resources (memory, processing cycles, communication bandwidth, and
storage) and compete with other users of those resources. For example, an auditor may request access
to a file with a program that will examine each record in the file and may lock other users out until the
process is complete. The processing could also require large amounts of network storage space at a time
when it is in short supply and could cause a server to crash. It is important to schedule such processing at
times when other system users will not be delayed or prevented from performing their work. Alternatively,
many audit organizations perform their audit analyses using files copied or archived from the live
production files.
CAATTs may also be large, powerful, or specialized enough to require a dedicated server for audit
purposes. A server may be needed to support the audit website, or just to assure the independence and
security required by audit functions. And, as evidenced by the list of software tools attached to this
document, there are more tools available than the amount of time an auditor may have to learn to use
those tools. So the need for software specialists to support internal auditing is increasing even as the
software is getting easier to use.

Risks associated with software tools and techniques

Software ease of use may also result in the implementation of features that unintentionally weaken
information security provisions. While software vendors may not be particularly open about their potential
weaknesses, a growing body of websites documents software weakness and available corrections. This
provides both positive and negative opportunities.
As weaknesses in software are discovered and documented, the vendors of those software products
develop corrections or patches that may be applied until the weakness is corrected in the next formal
release version of the software. However, many organizations do not apply such patches, for a variety of
reasons. Hackers know software frequently goes unpatched, so they search for particular versions of
software with known weaknesses. They may then launch an attack against that system using software
developed to exploit known weakness. Such software, called a "script," may require little or no knowledge
to use. The successful attack using a script may give the attacker unlimited (or root) access to the target
system. Normally, root privileges are reserved for system administrators and are closely monitored. Once
an attacker has root access they have virtually unlimited access to the system, and may also obtain
access privileges to other systems with an established trust relationship.
Another element contributing to risk in information systems and networks is the configuration of systems
as provided by vendors. Frequently systems are initially installed with the security and control features
turned off. System and network engineers and administrators must select the appropriate mix of control
features they need and turn them on when the system is installed. Sometimes security and control
features will conflict with features of other system components or may add considerable overhead to
system processing, such as through the use of system logging. When security components conflict with
operations, the typical response is to turn those components off. Unless the organization provides strong
security policy administration and/or auditing, management may be unaware security features are not
being used. Therefore, frequent assessment and monitoring are important elements of information
security management.
The Center for Internet Security (CIS) (see http://www.cisecurity.org, a not-for-profit organization) has
developed benchmarks for identifying the security features that should be activated for specific operating
environments, and publishes the specific settings for individual operating systems. These benchmarks are
available on their website. CIS also provides downloadable software to check system configurations
against the benchmark.
Electronic working papers
The capability to search for information in text, databases, or other audit records is giving auditors great
ability to coordinate their efforts and to examine findings from prior or concurrent audits. The ability to
require standardized audit forms and formats can improve both the quality and consistency of audit
working papers. The management of current and archived working papers in a centralized audit file or
database can make it easier for audit management to coordinate concurrent audits and assure they
consider findings from prior or related projects.
Expert systems provide an opportunity to add broad support and increased functionality to audit working
paper tools. For example, an expert system may evaluate responses to a questionnaire and automatically
generate links to additional related questions. Expert systems may also look at patterns in information,
findings, recommendations or related concurrent or previous audits, and provide reports indicating
potential related or systemic problem areas.
As audit work paper tools provide the ability to include supporting information other than text or numbers
such as pictures, sound, and video the methods for organizing and providing access to such
information must adapt accordingly. In future, auditors may discover that a great deal of information
needed in audit reviews may exist in forms other than text, numbers, or graphical characters.

A word of caution is in order: As you consider commercial solutions for managing electronic working
papers, consider the environment in which the software will operate. Some packages require
environments that may be inconsistent with the systems and networks maintained by the rest of the
organization. Consider also flexibility. Some packages may be limited in the options available for different
types of working papers that can be used and communicated among audit team members. Some
packages may need modifications to suit the needs of your organization. Modifications may result in
difficulty applying new releases of the software and/or may void the vendors warranty of features and
functionality. These considerations are certainly not unique to audit software tools and are part of the
complexity routinely managed by information services professionals and management.
Information retrieval and analysis
To sample or not to sample
Historically, auditors have relied on samples of transactions to perform their tests. With the use of
automated retrieval and analysis tools, it may be easier to assess all records than to evaluate a sample.
Furthermore, auditors can set parameters in software to identify all records meeting selection criteria. Full
selection of known error type records can eliminate the problem of estimating error rates. Instead, error
analysis can focus on those records with data that are outside the range of expected transaction values
but still within the limitations that define error conditions.
Actual sampling techniques may be applied at the time records are selected from the production system,
or all records of a given type may be selected and sampling or more detailed selection may be applied in
the analysis process.
Record selection criteria may be based on prior audits, but auditors should continuously assess
opportunities to improve audit coverage especially if this can be accomplished at reduced overall cost.
Automated selection and analysis tools can facilitate improvements, but will not automatically assure such
improvements.
Retrieval and analysis software
Identifying and accessing information
Information retrieval and analysis tools can present significant technical challenges to auditors as
information subject to audit may reside in diverse and distributed system types with varying degrees of
control and standardization. Data may be stored under the control of various machine types and operating
systems using differing formats; it may move across telecommunications environments using different
protocols; it may be stored or archived by various database management systems using fixed or variable
length fields or records and subject to differing database standards; and it may even reside in numerous
physical locations as in a distributed database or data warehousing environment. Particularly sensitive
data may only be available in encrypted form and may be subject to government regulations regarding its
transmission, storage, controlling software, encryption key management, and import / export or
transmission across national borders.
Many auditing departments use technical specialists to locate and evaluate data sources and provide the
software tools to extract data and convert it into a form that can be used by audit analytical tools. Because
there are so many forms and formats for information and so many proprietary standards for information
storage, and because information systems environments change frequently, it may be necessary to
maintain significant technical expertise among the audit team members responsible for using retrieval
software. People with such expertise may be difficult to recruit or afford, and providing training to audit
staff for such skills may make them highly marketable.
In some organizations or industries information is stored according to specified standards that do not
change frequently, and multiple audits may be performed on information in a common format. In such
cases libraries of information retrieval routines can be maintained, accessed, and executed by any
auditor. In other organizations the frequency of change may be greater than the frequency of audits and
preparation of retrieval software routines may preclude the use of pre-programmed routines.

Once information is stored in a form usable by audit analytical tools, auditors with varying degrees of
technical expertise may actually perform and review the results of analysis. Many ordinary office software
tools such as spreadsheets or databases may be able to access and analyze information stored in an
open database compliant (ODBC) format.
Some audit organizations not only maintain automated routines for information retrieval and analysis, but
they deploy such software via telecommunications to allow reviews of remote systems without the time
and expense of staff travel. Organizations with centralized controls and standards management are best
suited to remote auditing, but auditors may also use some of the same types of software as deployed by
hackers to assess security and control in distributed systems environments without centralized controls.
Information analysis
Accumulation of information about business data over a period of time may allow analysis software to
identify patterns, shifts, or trends in the data indicating changes in the business, the business
environment, the customer base, the economy, changing competition factors, etc. Such pattern analysis
may be important to business planning and competitive advantage, and may be performed by groups
outside of internal auditing. However, if audit analysis recognizes such patterns then the auditors may be
able to provide a valuable contribution to the organization.
Audit analysis of data patterns may be focused on shifts that indicate a need to redefine record selection
criteria, quality management mechanisms, error threshold monitoring, or review of records and
transactions that fall outside the normal realm of events (possibly defined in standard deviations). But
audit analysis can also target certain data patterns such as identification of artificial numbers. For
example, Benfords Law defines a natural distribution of numbers common to all large bodies of numbers.
In circumstances where individuals make up or modify numbers due to fraud or errors, the resulting set of
numbers will not follow Benfords Law and may be detected and investigated via audit analysis software.
(For more information on this subject there are several articles in ITAudit Forums archives. Mark Nigrini
wrote a series on Benfords Law and Digital Analysis published in the Emerging Issues department; and
Rich Lanza wrote an article on Continuous Monitoring published in the Audit Tools department.)
More common audit data analysis routines include matching employee data to customer or vendor
records, duplicate payments, payroll and overtime, approvals versus authorization levels, force codes,
system overrides, access authorities, telephone usage, and much, much more. Examples abound in
auditing literature.
Trends in information retrieval, analysis, and monitoring
A trend in auditor information retrieval and analysis is to include greater intelligence in auditing or
monitoring software embedded in business systems and networks. As auditors identify risk elements and
develop software to detect errors, suspicious transactions, or unusual data patterns, it is often a relatively
simple process to embed such tests or monitors into production systems. In these cases, auditors can
then be informed of errors or changes in data patterns soon after they occur throughout the operating life
of the system or monitor.
Auditors planning to deploy embedded system audit features can be identified as "users" of systems
under development. Rather than functioning on the design and development team only as control
specialists, they function as any other system user or interfacing system representative. The auditors
specify the record selection and data format criteria for embedded monitors, as well as any special
features such as logging, or the ability to modify, expand, or suspend audit monitoring.
For example, auditors may expect certain systems to process transactions at expected volumes or within
certain monetary ranges. Embedded monitors may alert the auditor by triggering an alarm if transactions
exceed expected threshold boundaries and may gather and store copies of the related transactions. The
auditor can then evaluate the data and determine if the fluctuations are normal or require additional

appraisal. In either case, the audit software may be provided additional logic or intelligence to enhance
such selection or appraisals in the future.
Typically, when audit monitors become more sophisticated than the tools used by managers responsible
for the systems, the managers will request that they also be provided such functionality. After all, no one
wants the auditors to come in asking questions about problems before management is even aware of the
problems. As management controls and monitoring tools become more sophisticated to match or exceed
the auditing tools, then auditors can shift their emphasis to areas of greater risk, or can increase the
sophistication or intelligence of their monitors. In either case, the overall control environment is enhanced.
In the future, the logic used by auditors to trace transactions and events forward and backward within
computer systems, networks, and files will also be embedded in sensitive systems. Then sensitive
transactions flowing through systems can carry with them embedded information indicating the source(s)
of the transactions and all routes taken through processing, networks, or files. Such "audit tags" will be
most useful in the case of monetary transactions such as payment processing or funds transfers and will
provide vital information needed to detect or deter fraud.
With the decreasing costs and new capabilities of information processing and storage systems and
media, it is becoming feasible to capture and archive sensitive information at all points of entry,
processing, transfer, or storage. The availability of "massive redundancy" in data management will enable
monitoring and analytical tools to track, in great detail, the changes applied to data throughout its life
cycle. Massive redundancy can also provide for data analysis using "voting" and other analytical or
statistical methods. Thus appraisals of information integrity in the future could be based on complex data
analysis and proceed to controls analysis only as anomalies are encountered. This is the opposite of how
traditional audit appraisals are applied and may require some process reengineering within the auditing
profession.
About this article
This article is extracted from a paper prepared for an "International Seminar on IT in Audit" hosted by the
National Audit Office of China (CNAO) September 16-21, 2001 in Beijing. The larger paper, titled
"Information Technology in Auditing," incorporates updated material from audit software articles originally
posted in the ITAudit Forum on September 1 and October 15, 1998. This article and the two subsequent
companion articles replace the older ones found in the ITAudit.org archives. An updated list of audit and
risk management software and related tools and services and their providers is also provided.

The IIAs work with the CIAO and PCIS continues with the PCIS supporting the
"National Plan for Information Systems Protection" and working to facilitate information
sharing across sectors of the critical infrastructures and extending outreach to other
nations to improve global security practices and help ensure protection of the global
economy. For more information, or to participate in this activity, contact Charles
Le Grand at The IIA.

Vol. 4, October 15, 2001

Audit Tools

Use of Computer-Assisted Audit Tools and Techniques

(CAATTs), Part 2
By Charles Le Grand, CIA, CISA, CDP
In Part 1, you saw that CAATTs can be classified into 10 groups. There, you reviewed the first two:
electronic working papers, and information retrieval and analysis. Here in Part 2, youll study the
remaining eight classifications: fraud detection, network security and performance, electronic commerce
and Internet security, continuous monitoring, audit reporting, database of audit history, computer-based
training, and time tracking.
Fraud detection
Areas most frequently identified by auditors for fraud detection include accounts payable, employee
payroll, expense reporting, and inventory management. Historically, auditors have looked for typical fraud
indicators such as duplicate payments for invoices or expense reports, invalid vendors, unusually high
payments or payments exceeding authority levels, payroll payments to former employees, or detectable
patterns in inventory "shrinkage." In recent years software has provided auditors with tools that can also
identify unexpected or unexplained patterns in data that may indicate fraud.
Benfords Law, as indicated in Part 1, provides several rules that apply to large bodies of numbers. One
example is the percentage of numbers in a population that will begin with the numerals "1" or "2." In
normal populations, fully 30% of the numbers will start with the numeral "1" and the percentage of
occurrence will decrease rapidly as the numerals increase from "2" through "9." By using analytical
software, such as ACL that can apply Benfords Law to a body of numbers, the auditor may be able to
detect fraudulent or "artificial" numbers because people making up or manipulating numbers typically do
not know about Benfords Law. A simple example of record types that may be detected through such
analysis is purchase orders generated for amounts just below an individuals authorization limit. Rather
than generating a purchase order for $25,000, which would require a higher level of approval, a person
may generate several purchase orders just under their $5,000 authorization limit. Data analysis would
detect an anomaly for an unusually large group of purchase order amounts beginning with "4."
Network security and performance
Network security software is typically used by network administrators. However, as the incidence of cyber
attacks perpetrated via networks has increased dramatically in the past few years, auditors have found
the need to add network security assessment software to their tool kits. Sometimes auditors will use the
same tools as those used by their network administrators. Other times auditors will use their own
specialized tools. Auditors may even use common hacker tools. In any case, it is important for auditors to
coordinate closely with network managers and administrators because tests and scans can adversely
impact network performance if used improperly. Improper use of network security analysis tools can also
cause a network to fail, ceasing operation until the network administrator is able to resolve the problem
and restore operation. Causing a network to crash could result in considerable costs as well as lost
productivity, revenues, and opportunity for an organization.
Among the most important features of any network are availability and performance. So auditors must
exercise extreme caution in ensuring their assessments and analysis do not impact network performance
or availability. (See Audits From Hell in ITAudit Forum .) There are also examples where an organizations
shareholder value was negatively impacted because the organization was the victim of a destructive
cyber attack or even a simple distributed denial of service (DDOS) attack.

In some cases network security assessment and analysis software may be provided free or at a low cost
by organizations hoping to sell security services or other products. However, auditors must be wary of
free software, particularly the variety known as freeware. It may be difficult or impossible to understand
the full functionality of such software, the impacts it may have on systems and networks, and the integrity
of its processing. Furthermore, without a legitimate vendor, there is no recourse for problems that may be
caused by the software, and user support may be difficult or impossible to obtain.
Network assessment and analysis software may be used to map the full extent of a network. Sometimes
a device on a network may be modified to act as a bridge or gateway to other networks. In such cases
network administration and management may not be aware of the full scope of the network and may
apply inappropriate security provisions. Network traffic analysis software, sometimes called sniffers or
supersniffers, may be used to analyze and/or capture messages or even individual keystrokes in
network traffic. Such tools, if improperly used, can violate security, confidentiality, and privacy rules, but
they can also be used to monitor and enforce information security policies and legal or regulatory
requirements.
More important than the tools used by auditors for network assessments and analyses are the tools that
make up the entire security environment for the organization and its networks. Network security tools
include firewalls, intrusion detection systems, worm and virus protection, backup and recovery, traffic and
pattern analysis, encryption, public key infrastructure (PKI) and certificate authority (CA) administration,
access control and monitoring, vulnerability assessments, and much more. It is pointless to focus on
individual components in a network security environment without addressing the full control system. For
example, firewalls can provide good controls but are ineffective if they do not properly apply security
policies or if their coverage is incomplete. And, virtually all security provisions can be circumvented by
social engineering if employees are not adequately instructed and monitored in applying good security
practices.
Electronic commerce and Internet security
Electronic commerce via the Internet has increased at an explosive pace in recent years. Most
organizations have implemented business-to-business (B2B) and business-to-consumer (B2C) ecommerce systems using Internet tools. Competition and opportunity are driving forces for this growth.
But rapid growth in an area of new technological developments inevitably introduces new problems and
escalates the significance of some older problems.
The Internet facilitates communications via e-mail. Today, e-mail is the standard for the rate of progress
and responsiveness for virtually every organization. Similarly, browsers and websites set the standard for
providing information about an organization and its products and services. And, in many cases, the
website is the vehicle for delivery of information, products, and services.
To be useful, information must be available, but this availability puts it at risk. Connectivity makes
information available when and where it is needed and is the nature of doing business today. Because
organizations are linked through the Internet and other public networks to suppliers, customers, and
business partners, they are also connected to virtually everyone else in the world. Connectivity exposes
information to risks outside the organizations control.
In the modern world, everything that business or government does with its information technology
becomes part of the global information infrastructure. Organizations must build infrastructures to a very
high standard. Attaching weak components to the infrastructure puts your organization as well as your
neighbors at risk. Responsible citizens will contribute only sound components to that cooperative
infrastructure. Therein lies the essence of the auditors involvement in providing assurance of the security
of information and systems operating in connection with the Internet.
E-commerce tools for auditors are just beginning to emerge. Generally, auditors are using the same tools
as systems administrators, information security professionals, and even hackers. An organization

concerned about its security may employ auditors or others to assess system security using tiger team
tactics authorized attempts to break into their systems. In many, if not most, cases such attacks are
successful and provide management with information about various ways outsiders can break into
systems or insiders can exploit system security weaknesses. Non-invasive tools are also used to probe
networks for security flaws that might be exploited. New tools are also being introduced that will evaluate
the configuration of security features in key network components such as the operating system, firewalls,
intrusion detection systems, virus protection systems, and more.
E-commerce tools also include encryption, public key infrastructures (PKI) and the related certification
authorities (CA) that facilitate the distribution and validation of encryption keys and related services. A key
feature of being able to conduct business over the Internet while being assured of a valid agreement and
protecting privacy is obtaining the services of third-party trusted agents. Assessment of PKI, CA, and
third-party trust features built into systems, networks, and business operations is beyond the capabilities
of most auditors today. Notable exceptions auditors who must be fully capable of addressing ecommerce systems, security, controls, and assurance auditing include those auditors working with
organizations who are the leaders in implementing Internet e-commerce systems. Such organizations
include major banks and related financial institutions, credit card providers and processing entities, large
manufacturing organizations engaged in B2B and/or B2C commerce, leading technology providers, and
similarly advanced organizations.
However, as previously noted, advancements in e-commerce are occurring at an accelerating pace. Ebusiness is becoming synonymous with business. The automated tools and techniques being developed
and deployed by the leaders today will become standard assurance and auditing techniques used by
auditors at all levels in the near future. A factor contributing to the increased capability of auditors in ecommerce will be the demands by boards of directors, insurers, and regulatory bodies for improved
assurance of effective and continuous information security.
Continuous monitoring
Continuous monitoring in systems and networks will be a byproduct of the increasing demand for
immediate and continuous access to reliable information by management, owners, investors, and
regulators of organizations of all types and sizes. The pervasive availability of electronic communications
drives the demand for reliable information and related assurance services.
Integrated accounting systems are rapidly becoming commonplace, and will soon be the established
basis for the expectation of timeliness in availability of financial information. Immediate financial reporting
and availability of information for comparison and analysis are becoming byproducts of integrated
applications across all areas of businesses and industries combining operational and financial
information in integrated databases and management reporting. The emergence of standards such as
extensible markup language (XML) and the related extensible business reporting language (XBRL) will
also help to accelerate the pace of increasing expectations for the availability of information and the
related assurance of its integrity.
As previously indicated, advancements in information monitoring and analysis are being accelerated both
by increasing demands for timely and accurate information, and by advances in technology that contribute
to the intelligence, capabilities, and timeliness of monitoring and analysis systems. Continuous monitoring
systems are not new, but they also cannot be considered widespread at this time. Nonetheless, the
advances in systems and the increasing expectations of information availability will ensure that
continuous monitoring and auditing systems will be the rule rather than the exception in the near future.
Audit reporting
Some audit tools today provide automatic linking between work performed, information gathered, auditor
assessments, and information used in or supporting audit reports. Intelligent work papers may note
answers in internal control questionnaires (ICQ) that indicate actual or potential weaknesses and
automatically prepare a section in the audit report to document the weakness and/or resolution of the
problem.

Audit reporting, too, can automatically provide information about sections of audits performed by
individual auditors as they are completed so the audit supervisor will know the ongoing status of audit
projects. Such reporting will also allow the supervisor to concentrate on audit processes that indicate
problems and/or provide additional resources in areas falling behind schedule.
The audit report can easily contain links to working papers, worksheets, graphs or other information that
will be automatically updated as data changes. Report files can be shared by audit team members and
management by implementing simple controls over access such as read-only access to those not
authorized to change the files.
Audit reports can be distributed in electronic format via e-mail, file transfer, or audit website. In such
cases, auditors must assure appropriate security, confidentiality, and access controls for such reports.
Encryption technology is rapidly developing and will become the standard mechanism for electronic
message integrity, sender and receiver authentication, and access control.
Database of audit history
The audit history database should provide a historical perspective for all audits on the plan or schedule.
Audit history can identify recurring or unresolved issues or problems, or indicate areas of risk.
Furthermore, many sections of audit work papers can be copied from prior files and updated to save
auditor time and effort.
Audit reports can be indexed by key words to facilitate review or searching, or may be searched in their
entirety depending on the techniques employed. Similarities in data patterns, audit findings, or
recommendations can be found using indexing or search technology, and can support expansion or
reduction of audit scope.
The technical delivery of the audit history database may be based in database management system
technology or may be delivered via a website. Regardless, it is also important to consider confidentiality of
audit information and provide access controls and other privacy and security techniques for files and
communications. Audit assessments of controls can represent a risk element because they could provide
information needed to identify control weaknesses.
Computer-based training
Embedded training and help features are included in most audit software tools today. Many software
providers and other organizations offer both generic and specific training for the use of software tools.
However, computer-based training (CBT) can span the broad realm of auditing, as well as activities
subject to auditing, and should not be limited by previous experience. Training can be informal and self
motivated, or it can be a formal element of audit administration providing feedback to the trainee as well
as to audit management.
In the context of CBT as an audit tool, it is most likely to be self motivated. It may be limited by the time
and tools available, the speed at which the tools operate, or the auditors energy, imagination, and
exposure to information. For example, if auditors do not have access to the World Wide Web, then they
cannot use it to search for information. If their access path is slow and/or expensive, then the time
requirements may quickly outpace the value received or reduce the auditors enthusiasm for such
learning. If traveling auditors do not have remote access to their central files or e-mail, then they cannot
search audit histories and cannot use a list server to seek input from others on a problem or question.
Ultimately, audit management, and of course the budget, will determine the tool set provided to auditors,
but the auditors themselves will determine how effectively the tools are used. Training should focus on
how to seek out and learn new information and approaches, not just on how to perform previously defined
tasks or use existing software features.

Time tracking
In some cases, it may be possible to direct internal system clocks to record the time auditors spend using
their computers and track that time to individual projects. It may also be relevant to record the time and
resources used by programs as they process for the purposes of individual audit projects. Eventually,
automated tracking of resources will become the norm, but today it is more likely to provide input only to
the time tracking and management processes.
An audit management system can provide detailed and summarized analyses of productivity and other
reporting parameters required to effectively manage an auditing department. Time tracking and reporting
can be elements of the project management system previously described, and can be used to evaluate
performance, estimate time requirements for scheduling, and relate critical skills to their most effective
deployment.
About this article
This article is extracted from a paper prepared for an "International Seminar on IT in Audit" hosted by the
National Audit Office of China (CNAO) September 16-21, 2001 in Beijing. The larger paper, titled
"Information Technology in Auditing," incorporates updated material from audit software articles originally
posted in the ITAudit Forum on September 1 and October 15, 1998. This article and the two subsequent
companion articles replace the older ones found in the ITAudit.org archives. An updated list of audit and
risk management software and related tools and services and their providers is also provided.
The IIAs work with the CIAO and PCIS continues with the PCIS supporting the "National Plan for
Information Systems Protection" and working to facilitate information sharing across sectors of the critical
infrastructures and extending outreach to other nations to improve global security practices and help
ensure protection of the global economy. For more information, or to participate in this activity,
contact Charles Le Grand at The IIA.

Guidelines for Requesting Data from Computer Systems

The following guidelines will save time and improve the chances for successfully obtaining and
testing computer data.
I. PLAN FOR THE REQUEST
Before requesting computer-generated data from IS departments, you should have the following:
1. A basic understanding of the computer system, including the purpose of the system, who uses
the system, what data elements (or fields) are available, what reports are routinely generated, and
what the data is used for.
2. An audit plan for reviewing or testing the data, including why you are testing the data, who
will test it, and what other files will be required.
3. The name and phone number of: 1) the person responsible for maintaining the system; and 2)
the person responsible for creating the computer data in response to your request.
To help understand the data in a computer system and identif~r exactly what data elements
(fields) you will need for testing, you must obtain and review the appropriate DATA
DICTIONARY or file layout. The dictionary should provide information such as the name,
source, purpose, and a narrative explanation of each data element in the system.
II. REQUEST THE DATA IN WRITING
Once you have the above information, you are ready to make your data request. The request
letter, usually signed by a manager or above, should include the name of the data elements
requested as they are identified in the data dictionary. Request only those data elements that are
relevant to your audit test; never request a copy of all the data elements in the system, unless
they are all needed to complete your planned test.
Your request letter should include:
~ The date by which you need to have the data;
~ The name and phone number of a person to contact if there are any questions regarding the
request;
~ A list of data parameters, such as specific transaction codes or a cut-off date for the data;
~ The format in which you want the data; for example, .dbf, .wkl, flat ASCII or EBCDIC files;
> The media on which the data is to be put; such as, disk, tape, download, etc.
~ The name and the phone number of the auditor requesting the data; and
~ The name and address to which the data should be sent.
It is very important that the client provide, in writing, the total number of records in the database
and the dollar amount (control totals) for all-important numeric fields.

Attachment I provides a list of technical specifications and documentation requirements that the
client should use when providing computer data to you. You should provide a copy of the
checklist to the client and request that they complete the list and forward it to you with the
computer data. Failure to include these specifications may cause a delay in processing the data.
III. AVOID POTENTIAL PROBLEMS
To reduce the probability of delays in processing your data, you should be aware of the following
general rules.
1. 1. Be cautious with print files. Print files are usually a copy of data listed on hard copy reports
that is stored as a computer file. Accordingly, they often contain data such as headers, footers and
subtotals that are shown on reports. If you do request a print file, you should also request some
pages of the hard copy report. Also remember that the data in the report file has already been
processed. Your test of the original data could be compromised if you limit yourself to just report
files.
2. Request fixed length files. Fixed length means that each record in the file has the same number
of characters. If the client cannot provide fixed length files, you may have to perform additional
steps to import the data into IDEA.
3. Verify that the client provided the required documentation. Incomplete documentation is often
the cause of problems in processing computer data. Accordingly, we recommend that you verify
that the client has provided all the needed documentation and that the data is in the format you
requested. If it is not, you should immediately contact the person responsible for providing the
data.
4. Microcomputer files can usually be imported into IDEA. However, there are a wide variety of
possible formats. Some formats can be troublesome. For this reason, if the client plans on giving
you data in microcomputer format, .dbf files (dbase format) are the easiest file formats to import
into IDEA.
ATTACHMENT I
TECHNICAL SPECIFICATIONS FOR COMPUTER DATA
1. Storage Medium:
3480, 3490, or 3490E Cartridge
9-track, 6,250 bytes per inch
Floppy diskette
CD-ROM
Network Server
Other (please explain):
Is the file compressed? Yes No

2. Data Specifications:
File Format:
EBCDIC
ASCII
Dbase
Other (please explain):__________________________________________
File Type:
Fixed Length File
Variable Length File
Field Separator______________________________
Record Delimiter(s)_____________________________________
String Encapsulator__________________________
3. Required Documentation:
a. Record layout that includes:
The beginning and ending position of each data element in the system;
Each data element's width; and
Each data element's type, such as character, numeric with sign embedded, or alphanumeric, etc.
b. Name and phone number of person(s) responsible for creating and providing the file.
c. File Name (Data Set Name)
d. Total number of records in file.
e. Control totals for important numeric fields.

Vol. 3, July 15, 2000

Standards

Auditing Online Computer Systems

By John Yu, CDP, FCGA

As previously reported, in March 2000 the International Audit Practice Committee (IAPC) of IFAC
released an exposure draft on four topics which form a supplement to ISA (International Standard on
Auditing) 401 Auditing in a Computer Information Systems Environment (CIS). The four topics are:
CIS Environments Stand-Alone Microcomputers
CIS Environments On-Line Computer Systems
CIS Environments Database Systems
Computer Assisted Audit Techniques
In a previous article, I reviewed the exposure draft on standalone microcomputers. In this article, Ill
review the exposure draft on On-Line Computer Systems.
Online computer systems
The exposure draft defines online computer systems as computer systems that enable users to access
data and programs directly through terminal devices This definition is sufficiently broad as to cover all
forms of online systems, including the traditional smart server/dumb terminal variety, as well as the
client/server variety because the definition covers all possibilities.
Contrary to the impression many people have, traditional dumb terminals still run a significant number of
the worlds CIS environment. These range from terminals used by travel agents and older generation of
point of sale (POS) terminals for many retail businesses, to terminals used in airline check-in counters
and those used to run most of the legacy systems used in many corporations. The exposure draft
describes two classes of terminals:

general purpose terminals such as basic keyboard/screen, intelligent terminals that

can perform a certain amount of data validation, and microcomputers
special purpose terminals such as POS devices, automated teller machines, and
voice response systems such as those used in telebanking

While these two classes cover a number of terminals used in online systems, they fail to recognize many
more modern (and advanced) terminals. The following are some examples of devices used in online
systems not covered by the definitions in the exposure draft:

biometric devices used for authentication (for a more detailed description of biometrics,
see Application of Biometrics)
network computers such as Suns JavaStation
Internet devices or e-appliances, such as personal digital assistants (PDAs), WebTV, iopener, various net-phones, and net-cars (for a more detailed description of eappliances, see What auditors should know about e-appliances)

All these devices operate in an online environment as terminals.

Types of online systems

The exposure draft suggests five types of online systems:

online/real time
online batch
online memo update
online inquiry
online download/upload

Online/real time systems are the classic online systems where transactions update the master file
immediately.
Online batch systems are those with online data capture but batch updates.
Online memo update is defined as On-line input with memo update processing, also known as shadow
update, combines on-line/real time processing with on-line batch processing. Individual transactions
immediately update a memo file containing information that has been extracted from the most recent
version of the master file. Inquiries are made from this memo file. These same transactions are added to
a transaction file for subsequent validation and updating of a master file on a batch mode. According to
this description, the transactions only update a copy of the master file, without affecting the actual master
file. The master file is affected only when the transactions are posted later. For all intents and purposes,
this form of online system is really a batch system.
Online inquiry systems restrict the user to perform queries only.
By the description in the exposure draft, online download/upload sounds like another variation of the
online memo update system where the memo file is a copy of the master file downloaded to the terminal.
After it is updated locally, it is then uploaded back to the original master file for updating.
The section on Characteristics of On-Line Computer Systems (paragraphs 18 to 22) seems to be a
hodge-podge of comments without any particular focus.
Internal control issues
As can be expected, this exposure draft devotes significant time to internal control issues. In fact, two
topics (Internal Control in an On-Line Computer System and Effect of On-Line Computer Systems on
the Accounting System and Related Internal Controls) are devoted to these issues. While the coverage
of internal control issues is reasonably comprehensive, the placement of certain paragraphs seems odd
at times. For example, under the second topic, I found a passing reference to risks of viruses. The issue
of risks associated with viruses should be given more prominent coverage under the general discussion
of internal controls rather than specifically on accounting system controls. Coverage of firewalls and
hacking should also be strengthened.
Effect of online systems on audit procedures
The exposure draft makes the point that it is more effective for the auditor to perform a preimplementation review of new on-line accounting applications than to review the applications after the
installation. Here, the focus is on on-line accounting applications, and seems rather narrow.
Increasingly, e-commerce businesses are relying heavily on online sales systems that are focused on the
sales and marketing side of the business, and yet such sales and marketing applications are more
important to the business than the accounting applications, which the auditors ignore to their own peril. In
any case, often, auditors need to audit online systems after they are implemented, playing no part in the
implementation.

Some reference should be made to auditing online transactions that involve third parties. This is
particularly the case with some e-commerce sites where the online credit card processing is handled by
an agent or service provider authorized by the bank external to the e-commerce site.
Overall, the exposure draft makes a good attempt to bring the standard up-to-date. The only major flaw is
that it has not gone far enough to deal with an increasingly complex online e-commerce environment that
provides auditors with new and special challenges.
The IAPC will accept comments and suggestions up to July 31, 2000.

Computer Assisted Audit Techniques

Readers' rating: 4 out of 5
By John Yu, CDP, FCGA

As I previously reported, in March 2000, the International Audit Practice Committee

(IAPC) of IFAC released an exposure draft on four topics which form a supplement to
ISA (International Standard on Auditing) 401 "Auditing in a Computer Information
Systems Environment (CIS)." The four topics are:

CIS Environments Stand-Alone Microcomputers

CIS Environments On-Line Computer Systems
CIS Environments Database Systems
Computer Assisted Audit Techniques

Authors note: Although this set of exposure drafts was published in March with comments due by July
31, 2000, a final version of these practice statements has not yet appeared on the IFAC Web site as of
early November 2000.
To review the first three articles on the exposure draft, see "Auditing Standalone Microcomputers",
"Auditing Online Computer Systems", and "Auditing Database Systems." In this article, youll learn about
the last topic, CAATs.
According to the exposure draft, the purpose of the statement on CAATs "is to provide guidance in the
use of Computer Assisted Audit Techniques (CAATs), which are techniques that use the computer as an
audit tool." The exposure draft "applies to all uses of CAATs involving a computer of any type or size."
As with the other three topics, this segment of the exposure draft reads like a tutorial on CAATs, devoting
a substantial amount of space describing the basics.
Description of CAATs
Paragraph 5 provides examples of where CAATs may be applied when performing various auditing
procedures. These include the traditional data analysis procedures, as well as the use of any computer
means in any aspect of an audit. To illustrate, one of the examples cited is the "creation of electronic
working papers by downloading the general ledger for audit testing." The "use of expert systems in the
design of audit programs and in audit planning and risk assessment" is also considered a form of CAAT.
However, in light of the importance of e-commerce in this day and age, at least one e-commerce example
should have been included in the list.
Paragraph 6 lists various CAAT tools, but these two paragraphs (this one and the preceding one) are
poorly organized. The list in Paragraph 6 consists of various types of computer programs that can be
used in CAATs (package programs, purpose-written programs, utility programs, and systems
management programs). The rest of the list consists of descriptions of various test data techniques. This
disjointed presentation is confusing. It is better to organize the material on test data techniques into its
own paragraph.
Paragraph 7 describes "evolving techniques that emanate from using the power and sophistication of
microcomputers, particularly laptop computers," then goes onto provide examples that do not
specifically apply to microcomputers and laptop computers. One of the techniques attributed to the power
and sophistication of microcomputers is "expert systems, which can design specific tests for use by the

auditor." You might well question the validity of this statement. In any case, the narrow distinction made
between "microcomputers" and "laptop computers" in this paragraph is an obsolete view of the computing
world. In the client-server model and the Application Service Provider (ASP) model, there is no need to
make the distinction between the workstation and the server, both forming an integral computing unit to
the user.
Manual tests
Paragraph 12 focuses on the impracticality of manual tests where there is lack of hard copy evidence.
This paragraph takes a negative approach and describes conditions under which manual tests cannot be
carried out, implying that there is no other choice but to use CAATs. This reflects old school thinking, in
which examining hard copy audit evidence is still considered the primary auditing method. Increasingly, as
organizations embrace the Internet as a means of conducting their business externally and internally,
there will be no hard copies. CAATs should be used by all auditors as a standard approach to auditing.
Using CAATs
Paragraphs 18 to 26 describe various steps required to use CAATs in a mainframe environment despite
earlier statements in the exposure draft describing CAATs as the use of any computing means in carrying
out an audit. Therefore, this narrow focus on mainframe environments where CAAT programs are run
against the auditees data files is inadequate when providing a full and accurate description of how CAATs
should be used.
Several references are made to the need for the cooperation of the auditees IT staff, stating the obvious.
But the exposure draft provides no guidance on how to proceed if cooperation is not forthcoming.
Paragraph 21 states that the "presence of the auditor is not necessarily required at the computer facility
during the running of a CAAT to ensure appropriate control procedures." This statement is puzzling. If the
auditor relies on the auditees staff to run CAAT procedures, what is there to prevent manipulation or
distortion of the results?
Using CAATs in small business computer environments
Paragraph 27 deals with the use of CAATs in a small computer environment. This paragraph, as it
currently stands, provides little guidance on what constitutes a "small computer environment." Another
example of incomplete guidance is "in cases where smaller volumes of data are processed, manual
methods may be more cost-effective." There is no direction on what constitutes "smaller volumes of data"
such that manual methods may be better.
Furthermore, the points raised in this paragraph again reveal antiquated thinking. To illustrate, one of the
points raised states "certain audit package programs may not operate on small computers, thus
restricting the auditors choice of CAATs." There are a number of powerful CAAT tools that can work with
virtually any type of data files from computers of any size. ACL is an example of such a tool.
Using CAATs in e-commerce environments
The exposure draft is silent on this very important area. More guidance should be provided. Some of the
audit techniques developed in the AICPA WebTrust program could be incorporated.
Dated approach
Of the four topics in the IAPC exposure draft on the supplement to ISA (International Standard on
Auditing) 401 "Auditing in a Computer Information Systems Environment (CIS)," the material on CAATs is
the most dated and requires a more innovative approach.