Systems and infrastructure life cycle management Phase One: Systems Planning Phase Two: Systems Analysis Phase

Three: Conceptual Design Phase Four: Systems Evaluation and Selection Phase Five: Detailed Design Phase Six: Programming and Testing Systems Phase Seven: Systems Implementation Phase Eight: Systems Maintenance

Systems Development Life Cycle and IT Audits1

.

In systems development, the temptation to skip certain prescribed tasks associated with documentation, combined with the fastpaced life of IT professionals, can create an environment that is not able to properly employ the best practices of systems development. However, the employment of best practices has proven over the years to provide returns in both efficiencies and effectiveness. In all types of audit, the employment of any set of best practices is generally seen by auditors as a positive impact on the quality of the information, systems or operations being audited. In the case of the systems development life cycle (SDLC), some practices provide additional benefits in terms of IT audits. Specifically, throughout the steps in the SDLC, documentation is being created that provides valuable potential sources of evidence for IT auditors. In other words, employing SDLC as it is prescribed in the industry is a control. In this article, the conventional phases of the SDLCand how each one can provide this potential evidencewill be discussed. Different groups use different lists of steps in the SDLC, but almost all agree on the same elements. Herein, a list of eight phases is used to demonstrate this process of analyzing an entitys SDLC. A summary of six of the eight phases and examples of related documentation are depicted in figure 1. Other documentation should exist; those contained in the figure are for illustrative purposes.

Phase One: Systems Planning

In phase one, systems are planned using a strategic approach. Executives and others evaluate the effectiveness of systems in terms of meeting the entitys mission and objectives. This process includes general guidelines for system selection and systems budgeting. Management develops a written long-term plan for systems that is strategic in nature. The plan will change in a few months, but much evidence exists that such planning pays dividends in terms of effective IT solutions over the long term.

This phase is similar to IT governance, and the two are quite compatible. Thus, the first thing an IT auditor would like to see is the implementation of IT governance activities. During this phase, several documents will be generated. They include the long-term plan, policies for selection of IT projects, and a long-term and short-term IT budget, as well as preliminary feasibility studies and project authorizations. Project proposals should have been documented when submitted to management, and a project schedule should exist that contains the approved projects (see figure 1).

The presence of these documents illustrates a structured, formal approach to systems development and, as such, illustrates an effective planning system for IT projects and systems. It also demonstrates a formal manner of approving IT projects. IT auditors will want to verify the presence of a systems planning phase (or IT governance activities) and take a sample of the documents to verify the effectiveness of that system. The same audit procedure will be true for all of the other seven phases and, therefore, will not be repeated in the narratives of phases two through eight.

Phase Two: Systems Analysis

In the systems analysis phase, IT professionals gather information requirements for the IT project. Facts and samples to be used in the IT project are gathered primarily from end users. A systems analyst or developer then processes the requirements, producing a document that summarizes the analysis of the IT project. The result is some kind of documentation, such as a systems analysis report (see figure 1). Other documentation should exist. In effect, systems analysis illustrates the entitys ability to be thorough with its systems development.

Phase Three: Conceptual Design

Next comes the conceptual design phase. In phase two, systems analysis, the requirements have been gathered and analyzed. Up to this point, the project is on paper and each user group has a slightly different view of what it should be. At this point, a conceptual design view is developed that encompasses all of the individual views.

A variety of possible documents could be the output of this phase. Figure 1 uses a data flow diagram (DFD), developed to a general level at this point, as an example. The point is that one or more of these documents should exist if the entity is following the SDLC thoroughly.

Phase Four: Systems Evaluation and Selection

During the systems evaluation and selection, managers and IT professionals choose among alternatives that satisfy the requirements developed in phases two and three, and meet the general guidelines and strategic policies of phase one. Part of the analysis of alternatives is to do a more exhaustive and detailed feasibility studyactually, several types of feasibility studies. A technical feasibility study examines whether the current IT infrastructure makes it feasible to implement a specific alternative. A legal feasibility study examines any legal ramifications of each alternative. An operational feasibility study determines if the current business processes, procedures and skills of employees are adequate to successfully implement the specific alternative. Last, a scheduled feasibility study relates to the firms ability to meet the proposed schedule for each alternative. Each of these should lead to a written feasibility report. Another aspect of this phase is a cost-benefit analysis. Quantifying tangible and intangible costs and benefits, an accountant should be able to determine the value of each alternative. This phase is associated with how to assess the value of IT. Finally, since a definitive choice among alternatives is being made, a selection report should be written to explain the reasons behind the choice and, possibly, include the costbenefit and feasibility studies.

Phase Five: Detailed Design

At this point, IT professionals have chosen the IT solution. The DFD design created in phase three is fleshed out; that is, details are developed and (hopefully) documented. Examples of the types of documentation created include use cases, Unified Modeling Language (UML) diagrams, entity relationship diagrams (ERDs), relational models and normalized data diagrams. Other systems design documents could also exist. IT professionals often do a walk-through of the software or system to see if any defects in the system can be detected during development. That walk-through should also be documented. To summarize this phase, a detailed design report should be written to explain the steps and procedures taken. It would also include the design documents referred to previously.

Phase Six: Programming and Testing Systems

For in-house development of applications, current best practices include the use of object-oriented (OO) programs and procedures. IT auditors should be interested in the IT programming shops choice of tools and procedures. Some businesses are locked into legacy systems and applications and, thus, would not be expected to use OO (e.g., banks). IT auditors would also be interested in programming flow charts as documentation. No element of SDLC is more important than systems testing. Perhaps none of the phases has been more criticized than testing for being absent or performed at a substandard level. Sometimes management will try to reduce the costs of an IT project by cutting out or reducing the testing. Sound testing includes several key factors. The testing should be done offline before being implemented online. Individual modules should be tested, but even if a module passes the test, it should be tested in the enterprise system offline before being employed. That is, the modules should be tested as stand-alone and then, in conjunction with other applications, tested systemwide. Test data and results should be kept, and end users should be involved in the testing. Figure 1 does not include this phase, but clearly the test results should be documented. The IT auditor will want to gain some assurance that proper testing of applications and systems has occurred before they are being put into operations.

Phase Seven: Systems Implementation

At this point, the system should be ready to deploy. The last step before deployment is a user acceptance sign-off. No system should be deployed without this acceptance. The user acceptance report should be included in the

documentation of this phase.

After deployment, however, the SDLC processes are not finished. One key step after implementation is to conduct a

postimplementation review. This reviews the cost-benefit report, traces actual costs and benefits, and sees how
accurate the projections were and if the project is able to produce an adequate return. The systems design is also reviewed and compared to the performance of the system to see if the information requirements processes (phases two and three) were performed adequately. In general, the time, costs and requirements are the three main elements of any IT project, and those elements should be benchmarked somehow. This step also reviews all of the system documentation to determine if it is adequate for the next phase: maintenance. If it is developed properly and according to SDLC best practices, it will be adequate. At a minimum, a user acceptance report and a postimplementation report should be documented during this phase.

Phase Eight: Systems Maintenance

IT professionals and IT auditors know that 80 percent of the costs and time spent on a software system, over its life cycle, occur after implementation. It is precisely for this reason that all of the previously mentioned SDLC documentation should be required. Obviously, the entity can leverage the 80 percent cost by providing excellent documentation. That is the place for the largest cost savings over the life of the system. It is also the argument against cutting corners during development by not documenting steps and the system. As changes occur, there should be change authorizations, change implementation and testing documents created during those changes. Testing during the maintenance phase should be able to use most of the original test data and test results, significantly reducing the time and effort necessary to adequately test the changes.

Conclusion
Employing the best practices of SDLC is not just a good idea in the IT industry; it serves as a control over systems development for IT auditors and provides documentation that the IT auditor can use to gain assurance over the adequacy and effectiveness of the entitys SDLC procedures. IT auditors are able to verify that SDLC best practices are operating effectively by examining documentation that should have been created during the various phases. Of course, IT auditors would use other means of verification, such as inquiry and checklists, but the presence of proper SDLC documentation illustrates the level of application of the best practices in SDLC. A review of a sample of the documents will provide evidence that the entity is using SDLC best practices, which provides some assurance that systems are being developed efficiently and effectively.

Auditing OS and Database Controls

To secure information effectively, it needs to be secured from all perceivable threats. The standard approach to information security has been to build layers of security that aim to control specific risks related to different components of a system. Figure 1 is a representation of a computer system, deliberately simplified to facilitate easier understanding of certain concepts.

The next few paragraphs may seem quite basic, but they are written with an audit and control focus, and the topic's understanding is important to the article. Essentially the data physically reside on a hard disk, which is a part of the hardware and is closely coupled with the processor and memory. The operating system envelops the hardware and interacts with all the input/output devices and connections outside of the computer. The operating system is the primary link between the software and the physical data and all attempts to read, write or manipulate the data must pass through the operating system. However, most end users of enterprise systems rarely interact with the operating systemnot by choice, but by good design. The users interact with the applications, e.g., the customer of a bank logs in directly to a screen that prompts for inputs required for withdrawal of monies or other transactions, and the store keeper logs into a menu that allows receipt of goods or issue of stocks. Application softwaresuch as enterprise resource planning, inventory management system, retail banking, financial accounting and invoicingare what users log into depending on their roles in the organization. Application software sits on top of the operating system (with a database management system, also on top of the OS). A user does not need to know what OS is being used, and the user's only interaction is with the application software.

Notwithstanding all of this, the IS auditor needs to be concerned about the operating system for the following reasons. The operating system sees all data on the disk as streams of bits in the records inside the files and folders. The operating system does not see the data relating to the basic pay of an employee as being significantly more or less sensitive than the employee's telephone number. It is the application software that understands the data from the business perspective; all business rules relating to the way the data can be manipulated are enforced through programs in the application software. For example, the application software does not allow a banking customer to modify the balance in the account, but only displays it and accepts a transaction. Good application software has controls designed to enforce all the validations and business rules relating to who interacts with which elements of the data and how. As long as the user stays within such an application, the user's actions are well controlled. Most application users log directly onto an application and, on exiting the application, are automatically logged out of the system. However, if a user is able to bypass the application and gain access to the operating system, then all the rules and controls in the application software become irrelevant. The OS views data not as basic pay, balance amount or stock value, but as a series of bits in a file. Once a user or an intruder gains access to the data through the operating system, the controls in the application software do not have any valuewhat the intruder can do to the data is dependent on the controls in the operating system only. Therefore, it is necessary to review whether adequate controls have been enabled in the OS. Auditing OS Every operating system includes a set of security features and vulnerabilities, which varies from OS to OS and sometimes between versions. The security features are designed in such a way that they can be turned on or off and set to high security or low security, depending on the purpose for which the user intends to use the OS. In most cases, the default settings are not designed for high security. It is up to the user to enable the security features to the desired level of security for that installation. The process of auditing OS security includes evaluating whether the security features have been enabled and parameters have been set to values consistent with the security policy of the organization, and verifying that all users of the system (user IDs) have appropriate privileges to the various resources and data held in the system. The audit of OS security requires the auditor to have a good knowledge of the various features of the operating system's security in detail. The better an auditor knows the system administration details for the OS, the more effective the audit will be. The audit of the evaluation of security parameters in the OS involves logging into the system and seeing the values on the system or running a few commands to find these values. Some of the most common security parameters that can be evaluated are password rules, such as: minimum password length, password history, password required, compulsory password aging, lock-out on unsuccessful logins, login station and time restrictions. The other areas of scrutiny are whether the logging of certain events, such as unsuccessful login attempts, has been enabled or whether the superuser password is held by the appropriate person. Other OS/version-specific parameters also have to be verified. Another area of scrutiny is to ascertain whether access privileges given to various users are appropriate. The first step is to ascertain what data/systems are on the server and how critical and sensitive they are. From this information, the auditor can get an idea of who should have access to what. Next, the auditor should obtain the list of user IDs in the system and map

these with actual users. Then, the auditor has to determine for each user what the permissions and privileges to the different resources/data are in the system. There are different methods, for example, commands for ascertaining this from the system for different OS. Another way is to determine for a given critical piece of data who the users with access are, and whether their access is appropriate. Besides application systems, many servers are used as file and print servers acting as a common repository for data for many users. In such cases, a review of the OS security to determine appropriate access for each user to his/her data is very important. Another point for examination pertains to the network. With all computers intricately connected to the internal and external networks, the network-related vulnerabilities of such systems also need to be covered in reviews, although they are even more specialized. Through suitable use of tools, the auditor should determine whether the services that are open and running in the server (such as FTP, Telnet, HTTP) or ports are only those that really are required. If the review is being done on a system that is hosting a web server or a firewall, the evaluation must be done by an expert. The purpose of this article is to focus on the concepts and need for the audit of OS and not to provide detailed guidelines or checklists for doing the same. Such guidelines or checklists are specific in technical detail to different OS. Many professional audit firms develop, through their own research, guidelines and work procedures for such technical audits. However, such checklists also have been published on various web sites by other professionals, and today it is very simple to download audit checklists for a variety of platforms using a search engine. Some books specific to OS have been published by ISACA; articles on the subject also are available in some back issues of this Journal. Some checklists are available in K-NET, ISACA's global knowledge network, at www.isaca.org/knet. When doing such audits for the first time, it is good to take the system administrator into confidence and keep him/her informed while running the commands and queries. Other tools are available to perform such audits. These tools take as inputs the security policy definitions, run commands to extract the parameter values from the system, perform a comparison and issue a draft report. Auditing Databases The current trends in application software design include frequent use of a database management system that actually handles data manipulation inside its tables, rather than it being done by the OS itself in files. The DBMS acts as a layer between the application software and the OS. The application passes on the instructions for manipulating data, which are executed by the DBMS following the integrity rules and constraints built into the database definitions. However, using a utility such as a text editor in the OS, the data in the DBMS can be manipulated directly, without the application. This can be done by using DBMS utilities and features, such as SQL (Structured Query Language)if the user can gain access to the DBMS. Hence, it is necessary to review security in the DBMS through a review of user IDs, the privileges associated with each ID and factors such as whether default-shipped passwords that are common knowledge have been modified/disabled. The procedures and exact commands to be used for carrying out the reviews will be specific to DBMS, and it is possible to obtain such checklists from the web and through books published by ISACA. Client-server and Web-based Applications There has been a significant change in the design of applications as well as their interaction directly with the OS since the advent of client-server and web-based applications. In such cases, users do not need to log in as users of the OS directly, but only need to connect to the database as a predefined user. While reviewing the OS in such systems, the auditor is likely to find there are just one or two user IDs created in the OS or in the database. While the user executes an icon in the desktop, a string containing the database user ID and password is communicated to the database, and a connection is made. It is necessary for the auditor to examine such scripts and evaluate their security. The way web-based applications connect to databases also is slightly different. The auditor should know that, in such scenarios, the older methods of audit do not remain fully effective, and therefore, he/she needs to adopt more current and appropriate methods. Conclusion

Auditing OS and database security is a key element of the total IS audit. Any deficiencies in OS and database security can nullify all of the security and controls that have been designed carefully in the applications. Hence, it is necessary to carry out reviews of the OS and database for all critical applications and the servers that hold sensitive information.

Generalized Audit Software: Effective and Efficient Tool for Today's IT Audits
By Tommie Singleton, Ph.D., CISA, CMA, CPA, CITP Volume 2, 2006

Because the author's experience and knowledge of GAS is primarily limited to ACL, that is the software used to demonstrate the value of implementing GAS in this article. For more software products, see the exhibit in the Bagranoff and Henry article, cited in footnote 3, and the list of GAS in the Sayana article, cited in footnote 1. Experts say that generalized audit software (GAS) is the most common computer-assisted audit tool (CAAT) used in recent years. There are many reasons today for IT auditors to use a GAS, but to quote an article from this Journal, "Performing audits without using information technology is hardly an option."1 This article will inform IT auditors of the profitable return on learning and using GAS. A number of issues motivate IT auditors to use GAS products, such as ACL, CA's Easytrieve, Statistical Analysis System (SAS), Statistical Package for Social Sciences (SPSS) and IDEA. First, there is the focus on fraud in recent years. According to the Association of Certified Fraud Examiners (ACFE) and its 2004 "Report to the Nation" survey on fraud, more than 60 percent of all frauds are detected either by a tip or by accident. While internal audit has moved up on the list, there is clearly room for more proactive antifraud programs. One of the best ways to be proactive is to use a GAS to develop a cornucopia of computerized antifraud audit procedures that are run regularly against organizational databases. Second, there is the issue of US Sarbanes-Oxley Act section 404. In a Journal article on Sarbanes-Oxley software, several GAS software products are included in the list provided in the authors' exhibit on data manipulation software.2 This indicates that GAS can be useful in testing internal controls embedded in information systems. A third issue motivating auditors to use GAS software is that the demands on IT and internal auditors are increasing. Auditors will need to become more efficient to fulfill all of the responsibilities and tasks assigned them, and using GAS is one way to do so. Therefore, IT auditors in the early stages of their careers could leverage their time and abilities into more productivity by becoming at least competent in a GAS product. Having a moderate level of knowledge in using a GAS, for example, can be useful in a variety of duties, such as fighting fraud, Sarbanes-Oxley compliance and everyday audits. Also, the more proficient an IT auditor becomes, the more valuable he/she becomes to the organization. This article encourages IT auditors to learn how to use ACL or a similar GAS. Benefits of Using a GAS The benefits of using a GAS have been explained by others, but a review of the benefits here will hopefully generate motivation to become more knowledgeable about GAS. As many others have pointed out, using a GAS such as ACL means the auditor does not review a sample of the data, but rather reviews or examines 100 percent of the data and transactions. This difference is not trivial. Some fraudsters have not bothered to conceal their fraud because they assume that the transactions involved have little chance of being picked in a statistical random sample. For example, in one fraud, the fraudster had written approximately 50 checks to himself out of organizational accounts (i.e., check tampering). When he was finally caught, he was asked why he did not try to conceal the fraud. He simply said he doubted that one of those checks would ever show up in a statistical random sample because the organization wrote thousands of checks each year. He was right and got caught without the benefit of those checks. All frauds that are "on the books" have the potential of being discovered by using, for examle, ACL effectively, because there is some kind of evidence in the database and the transactional data and ACL can be used to examine 100 percent of the data. Using ACL empowers the auditor to possibly have a better sense of direction in his/her audit procedures. Using ACL to analyze transactions, or data mine, is a lot like peeling an onion. The auditor will perform some audit procedures to gain an understanding of the data (e.g., using PROFILE, STATISTICS commands in ACL). During these procedures, the conscientious, trained auditor may spot an anomaly or red flag (e.g., negative amounts where there should be none). At that point, the auditor is focusing directly upon certain suspicious data and/or transactions. In ACL, these transactions are usually linked via the table or chart in the display window, so employing drill-down procedures is extremely simple when the auditor

needs them. The same is true as the auditor progresses through more precision audit procedures (e.g., using FILTER for certain anomalies or red flags). The data in ACL are locked down as read-only. There is no chance for the auditor to inadvertently change the data. This inadvertent risk is much higher for IT auditors who choose to use a spreadsheet for analyzing and presenting transactions. While one can lock cells or sheets in Microsoft Excel, there is still the possibility of human error. It is almost nonexistent in ACL. The commands in ACL are auditor-friendly. ACL commands are compatible with the average IT auditor's understanding, experience, training and education. It is fairly easy to grasp what a command will do once it is explained adequately. For example, auditors know what it means to look for gaps or duplicates in numbers (invoices, checks, etc.). The learning curve, therefore, is reasonably short. At most, the IT auditor will need training and encouragement to "think outside the box" with those commands. Most IT auditors will pick up on this flexibility without additional training. The ACL commands are effective in a variety of applications other than the obvious. For example, the AGE command is obviously useful in generating an aged trail balance. However, it is really a measure between dates, so it could be used to do other antifraud procedures. For example, it can send confirmations to credit card users for a recent charge where the card had been inactive for a certain number of months (six or 12 or whatever is appropriate) or be used in conjunction with the CLASSIFY command to measure the number of days between receipt of invoices and payment of invoices by vendor (in shell company, pass-through vendors and other fraudulent disbursement schemes, the fraudster tends to make sure the phony invoices are paid quicker than normal invoices). Another example is the CLASSIFY command itself. It is normally used to subtotal amounts and the number of invoices for vendors or some similar application on other data files. However, one could use CLASSIFY to examine the number of credit memos by authorizing party or key-punch personnel. Because credit memos are a relatively common method of concealing a fraud, if a fraud is being perpetrated and the fraudster is using credit memos to hide the fraud, that person has an inordinate number of credit memos compared to everyone else. This anomaly would be evident by running CLASSIFY on a credit memo file. The possibilities are limited only by the IT auditor's imagination. Additionally, ACL automatically records all of the commands that are run and the results of the procedures in its log, so the LOG feature becomes a way to automate much of the working papers an IT auditor will need to generate in most audits. ACL has a simple means to export that log to a word processor or other types of files, even selectively choosing which procedures to export. The most compelling benefit in learning to use ACL may be the BATCH feature. As the IT auditor develops audit procedures to run in ACL, he/she can put the various routines together in a batch (similar to a macro). Next time, the IT auditor can run one command (push a button), and all of those procedures will run on autopilot, and ACL will dump the commands and results into the log. That feature provides a great opportunity to be efficient over time. The first year might take some time, but future years will be much quicker. In addition, as new procedures come into being, they are simply added to the BATCH and will run with all the others next time around. There is a great opportunity for sharing among all the auditors in the same entity, thus expanding upon the batch procedures of various teams or among different areas of audit. In summary, there are many benefits to using ACLit just becomes a matter of budgeting for the cost of the software and implementing the use of ACL effectively. Implementation There are several ways for one to become moderately proficient in a GAS. Most IT auditors know how to use Excel and are fairly competent at it. With a little training in GAS in general, the IT auditor could first use an intermediate product, such as Information Active's Active Data or Active Audit tools.3 These tools are plug-ins to Excel; thus, the learning curve is fairly short. They contain many of the same commands, occasionally by another name, as those mentioned previously (e.g., GAPS and DUPLICATES). This approach uses a "gear up" methodology. However, there are drawbacks to Excel in terms of integrity, the amount of data that can be handled and the limited power it has, even with Information Active products. But it might serve as an effective interim means for some IT auditors, particularly for reasons of cost constraints. In fact, for some smaller audit units, it might be the ultimate means and not just an intermediate one. With some training, the IT auditor can become moderately proficient in GAS in a relatively short period of time. Of course, it might be better to get the training in GAS, some training in a specific product, and jump straight into the specific product especially if the internal audit shop or audit entity already has the product. Keys to Success

There are some keys to success for the internal audit (IA) shop or audit entity to make it possible for the IT auditors to effectively use GAS. First, the audit entity needs to identify a champion for the implementation. Research is replete with evidence that technology innovations and implementations need a champion to be successful. A champion is simply the person with the ability to motivate, supervise and generally make sure the technology is employed and becomes successful. In an internal audit shop, the IT audit manager could take on that role. Second, there should be general training for the audit staff regarding GAS. Next, the champion or IT audit manager should identify the power users of GAS. These people are given specific training if necessary, but they become the leaders of implementing the chosen GAS product. They set up the serversthat is, they would build the appropriate data files from the operational system and make them available to all the auditors. They also write or assist auditors in writing batches. They could also conduct ongoing in-house training on the product. If necessary, a consultant can be brought in to assist the power users in developing the server and customized services. While these things are outside the control of most IT auditors, they are facilitating or empowering approaches to effectively using GAS. Conclusion When thinking about one's career as an IT auditor, perhaps no other skill or ability is as valuable as being an expert at using GAS. Such expertise can be used in a variety of ways, including regular financial audits, operational audits, SarbanesOxley-related tests and antifraud audit programs. In fact, it can possibly make an IT auditor indispensable. Endnotes
1 2

Sayana, S. Anantha; "Using CAATs to Support IS Audit," Information Systems Control Journal, vol. 1, 2003 Bagranoff, Nancy A.; Laurie Henry; "Choosing and Using Sarbanes-Oxley Software," Information Systems Control Journal, vol. 2, 2005 3 See www.informationactive.com Tommie W. Singleton, Ph.D., CISA, CMA, CPA, CITP is an assistant professor of information systems at the University of Alabama at Birmingham (USA), Marshall IS Scholar, and director of the Forensic Accounting Program. Prior to obtaining his doctorate in accountancy from the University of Mississippi (USA) in 1995, Singleton was president of a small value-added dealer of accounting information systems using microcomputers. In 1999, the Alabama Society of CPAs awarded Singleton the 1998-1999 Innovative User of Technology Award. Singleton is the ISACA academic advocate at the University of Alabama at Birmingham. His publications on fraud, IT/IS, IT auditing and IT governance have appeared in numerous journals, including the Information Systems Control Journal.

IT Audit Basics

Auditing Security and Privacy in ERP Applications

By S. Anantha Sayana, CISM, CISA, CIA Volume 4, 2004 Enterprise resource planning (ERP) applications have over the years become the foundation application for many large and medium enterprises all over the world. ERP systems form the core transaction processing platform for most enterprises in manufacturing and related industries. The business processes and transactions of the company happen in the ERP system. The ERP is an integrated system that holds all the data pertaining to these business processes from the beginning to the end of the chain in a unified database. ERP systems, assisted by networks and central databases, have greatly improved efficiencies of transaction processing by eliminating barriers of geography and departments. Does all of this impact internal controls? Yes, substantially. The integrated nature of the ERP enables data entered at one part of the process cycle to be carried forward to the next part of the cycle for further processing. The acceptance of the validity of the earlier part of the data is implicit, and there is often no reverification at different stages. An example is provided in figure 1 (each box denotes a process and the department where it occurs).

Every one of these processes would have originated from different departments and may be located anywhere, but the ERP sees all the data from the common database. At process number 6, the ERP system picks up the purchase order data entered at process number 2, the goods received and accepted from process number 3, and the supplier's claims from process number 5. It then matches them all, calculates the payment due after applying all the commercial terms, and arrives at a payment amount and date. The only data that it seeks (even this can be automatically configured) are approvals for payment and the bank account from which the payment is to be made. In most stable ERP scenarios, there is no verification from base documents at this stage. Besides this, many other activities performed earlier by other systems may also be done automatically by the ERP as indicated in box 7. This is only one example. Similar process chains exist for almost all activities, such as the sales cycle or the manufacturing cycle. As is indicated, the process in box 1 can also be the result of another automated process called the material requirement planning (MRP) run that might have taken inputs from the sales forecast, manufacturing capacities, etc. In this scenario, the impact on controls is that there is no room for checking along the way and again at the end. Two major requirements of controls emerge from this significant feature of ERP systems:

1.

Every piece of data needs to be authentic and accurate at all stages of the process cycle. An error or a malicious entry at any of the stages would be carried forward all the way (subject to some validations that can be built). This requires clear definitions of who should do what and segregation of duties between people doing those duties enforced through access control. Fortunately, the major ERPs provide excellent features for defining access control to a great degree of granularity.

2.

Many automatic processes and updates (postings) can happen without human intervention. These are based on the configurations in the ERP. Therefore, it is necessary that all these configurations be made and maintained correctly and in line with the control requirements of the business. Most ERPs are designed as generic solutions that can operate in many countries and businesses catering to different requirements. Therefore, there is a lot of parameterization, and many parts of the system are configurable. While this is a desirable feature offering options during implementation, it can also be the source of internal control weaknesses allowing options that were hitherto completely prohibited. Approach to Auditing ERPs The following would be typical prerequisites for carrying out an audit for assessing security and privacy issues in ERP:

Understand the business and the risks specific to that business and its operations. This is the first basic step, not only in ERP audit, but in any kind of audit. Understand the enterprise structure. The complexity of ERP arises from the fact that it attempts to encompass the entire business cycle of the enterprise, covering all major functions and all parts of the organization in its wake, including financial accounting, human resources and management information. The mapping of the various organization units is reflected in the enterprise structure in the ERP. An accurate understanding of the enterprise structure is the first step to carrying out an audit.

Obtain access to the system. The audit of an ERP cannot be done by having someone generate reports and show them to the auditor. The information systems (IS) auditor should verify the various settings and data in the system by seeing them in the "live" system. For this, the auditor should obtain a user ID in the production system with read-only access to all modules and features. Needless to say, the auditor would need to have a good grasp of SAP and use the access with due care. Major Areas of Audit Relating to Security and Privacy The following are the major areas of audit:

Evaluating access controlAll the data in the ERP are in one single database. Theoretically, it is possible with suitable access to see or modify any part of the data relating to financials, materials, human resources or sales. In a centralized system connected through a network, access to the ERP is possible from any part of the network. Like any other application, access to the data has to be secured not only at the application level, but also at the operating system, database and network levels through suitable controls. The IS auditor should carry out a review of the network, OS and RDBMS before evaluating the access controls in the application. Access control evaluation in an ERP can be a complex exercise and requires good skills with the particular ERP. Most ERPs control access though menu options, screens and authorization objects, and activity options, such as add, modify, view and delete. To simplify the access control process, many ERPs use the concept of roles, assigning standard authorizations to the roles and attaching the roles to users as per their job descriptions and duties required to be performed. As ERPs are large systems, they have many users.

User ID scrutiny and evaluationA good first step for auditing access control is to obtain a list of authorized users and their privileges. The IS auditor should also obtain the list of standard roles and the authorizations required for the roles. Having obtained this information, the IS auditor can get the list of users from the system through suitable menu options. (Each ERP has its own procedure for obtaining this information. Such information would be part of the audit program available with audit firms or can be obtained by discussions with the ERP consultants or by referring to books.) This can be verified with the list of authorized users. Any mismatch between these lists needs to be investigated. It is useful at this stage to obtain from the system a list of user IDs created and deleted during the period of audit and to find the appropriate explanations and supporting approvals for all these transactions. The next step is to ascertain that all the authorized users have the appropriate privileges in line with their job responsibilities. To do this, the auditor should examine the roles that have been assigned to the various users, and ensure that the roles are in sync with their current job responsibilities. The next step is to identify from the system the authorizations and privileges associated with each of the roles in use, to ensure that roles, as defined in the system, are in line with what the role is expected to be. These roles could be called by other terminology, such as profiles, in different ERPs. Some ERPs provide for a level of granularity that can be mind-boggling. To cite an example, it would be possible to define a situation where a storekeeper can create, but not update or delete, goods received notes and only for a certain category of goods at a specified location of the store. Privileges of create, update, delete and view can be associated with every object based on the field values. While this is great news to fulfill the very stringent needs of security and access

control, this can easily get out of hand if the entire access control is not properly mapped, planned and documented meticulously, using all aggregation options at the appropriate levels. It is also good to approach access control evaluation from another perspective, i.e., from critical functions and activities that would be required to be carried out only by specific authorized personnel. This would require a thorough knowledge of the ERP system to identify certain critical transactions. The IS auditor can query the system to determine the users who have authorization to execute such activities and options. The IS auditor can evaluate whether the list is aligned with the approved policies of the company. Examples of these activities include the ability to execute certain sensitive reports, close or open financial accounting periods for postings, and make changes to certain sensitive masters. In addition to access control, most ERPs may also require explicit authorizations to approve certain kinds of actions. These authorizations may be associated with monetary values and other special conditions, such as approving discounts, sales, purchases and some changes. These should be verified from the system directly and compared with the organization's policies for such activities.

Verification and evaluation of configurations relating to business processesThis is another major area of audit for an ERP system. The process flow in every business activity, and the possible options for carrying out these activities are decided by the configurations in the system. Such configurations exist literally in every process and module, and the process of evaluating them can be a long and tedious job for the auditor. The best way to do this is to approach it from the business risk perspective and identify the configurations pertaining to those processes. Each of these configurations can be seen on the system by executing certain commands or following a certain path in the menu. An audit program that lists these and the possible values and options of these is a necessary tool, without which it would be difficult to carry out this evaluation. The recent books published by ISACA on the security, audit and control of SAP, Oracle and PeopleSoft can be useful for doing such reviews. IS auditors can use these programs as the baseline and customize them to their environment based on their experience, with help from the functional consultants.

Change managementThe other factor that can influence security is change management. The way modifications are done to the programs and the configurations, the effective segregation between the development and production environments, the processes for testing, quality assurance and migration need to be reviewed by the auditor. InterfacesERPs invariably need to send or receive data from other systems. Interfaces can be simple batch uploads of data or real-time data movement from multiple systems to and from the ERP through an integration middleware platform. An interface has the potential to become a weak point that can compromise security. Audit scrutiny of interfaces is one of the important aspects of ERP security reviews. At a simple level, controls should exist to validate data before upload and verify accuracy and integrity through control totals and logs, but a review of interfaces through middlware would be a specialist exercise.

PrivacyERPs are such a large repository of different kinds of data that they can impact privacy issues in a significant way. Most ERPs have a human resources and personnel module, and such modules handle a lot of data about the employees of the company. The ERP can also hold information about customers, suppliers and partners. Privacy laws have been enacted in many countries, and they all have their own flavors. ERPs may operate across countries in a multinational company and deal with data about employees and customers from different parts of the world. ERPs or their CRM extensions can also collect other information, such as credit card or credit rating information, which is impacted by privacy requirements. Such companies have to take due cognizance of the laws pertaining to privacy in each of the countries in which they operate or have dealings. Without getting into individual laws, general requirements may be grouped under a few broad requirements:

Securing personal information from unauthorized access, ensuring its protection through the life cycle of the information and ensuring its effective destruction when it is no longer required Catering to the rights of the subject about whom data has been gathered, processed and stored Not using personal information for purposes other than those for which the information is gathered or maintained

Many of these requirements can be met only if access controls are rigorously implemented and enforced. The way to address audit is first to identify the legislation, statute or rules that are applicable. This itself may be a formidable task, spanning across countries, and there may also be industry-specific regulatory requirements. The second step is to verify what kind of personal data are stored in the ERP and then go through the audit relating to securing them. Conclusion

Many factors need to be reviewed during an audit to ensure security and privacy in an ERP system. However, the most crucial among these is maintaining the access controls and configurations, as these are widespread and can have a significant impact. These are constantly subject to change; therefore, maintaining them is a challenge. In this context, it is necessary to carry out the audit of all aspects of an ERP comprehensively at a post-implementation review. Subsequently, the review of access controls and configurations should be carried out regularly, at least once in six months. The ideal situation would be to move this kind of audit to a highly automated process using tools to perform a continuous monitoring audit.

3.14 AUDITING APPLICATION CONTROLS

The IS auditor's tasks include the following: Identifying the significant application components and the flow of transactions through the system, and gaining a detailed understanding of the application by reviewing the available documentation and interviewing appropriate personnel Identifying the application control strengths and evaluating the impact of the control weaknesses to develop a testing strategy by analyzing the accumulated information Reviewing application system documentation to provide an understanding of the functionality of the application. In many casesmainly in large sysems or packaged softwareit is not feasible to review the whole application documentation. Thus a selected review should be performed. If an application is vendor supplied, technical and user manuals should be reviewed. Any changes to applications should be documented properly. The following documentation should be reviewed to gain an understanding of an application's development: o System development methodology documentsThese documents include costbenefit analysis and user requirements. o Functional design specificationsThis document provides a detailed explanation of the application. An understanding of key control points should be noted during review of the design specifications. o Program changesDocumentation of any program change should be available for review. Any change should provide evidence of authorization and should be cross-referenced to source code. o User manualsA review of the user manuals provides the foundation for understanding how the user is utilizing the application. Often control weaknesses can be noted from the review of this document. o Technical reference documentationThis documentation includes any vendorsupplied technical manuals for purchased applications in addition to any in-house documentation. Access rules and logic usually are included in these documents. o (go to data integrity testing)

3.14.1 FLOW OF TRANSACTIONS THROUGH THE SYSTEM

A transaction flowchart provides information regarding key processing controls. Points where transactions are entered, processed and posted should be reviewed for control weaknesses.

3.14.2 RISK ASSESSMENT MODEL TO ANALYZE APPLICATION CONTROLS

Risk assessment, as discussed in chapter 1, provides information relating to the inherent risk of an application. A risk assessment model can be based on many factors, which may include a combination of the following: The quality of internal controls

Economic conditions Recent accounting system changes Time elapsed since last audit Complexity of operations Changes in operations/environment Recent changes in key positions Time in existence Competitive environment Assets at risk Prior audit results Staff turnover Transaction volume Regulatory agency impact Monetary volume Sensitivity of transactions Impact of application failure

3.14.3 OBSERVING AND TESTING USER PERFORMING PROCEDURES

Some of the user procedures that should be observed and tested include: Separation of dutiesEnsures that no individual has the capability of performing more than one of the following processes: origination, authorization, verification or distribution. Observation and review of job descriptions and review of authorization levels and procedures may provide information regarding the existence and enforcement of separation of duties. Authorization of inputEvidence of input authorization can be achieved via written authorization on input documents or with the use of unique passwords. One may test this by looking through a sampling of input documents for proper authorization or reviewing computeraccess rules. Supervisor overrides of data validation and editing should be reviewed to ensure that automatic logging occurs. This override activity report should be tested for evidence of managerial review. Excessive overrides may indicate the need for modification of validation and editing routines to improve efficiency. BalancingPerformed to verify that run-to-run control totals and other application totals are reconciled on a timely basis. This may be tested by independent balancing or reviewing past reconciliations. Error control and correctionIn the form of reports that provide evidence of appropriate review, research, timely correction and resubmission. Input errors and rejections should be reviewed prior to resubmission. Managerial review and authorization of corrections should be evidenced. Testing of this effort can be achieved by retabulating or reviewing past error corrections. Distribution of reportsCritical output reports should be produced and maintained in a secure area and distributed in an authorized manner. The distribution process can be tested by observation and review of distribution output logs. Access to online output reports should be restricted. Online access may be tested through a review of the access rules or by monitoring user output. Review and testing of access authorizations and capabilitiesAccess control tables provide information regarding access levels by individuals. Access should be based on job descriptions and should provide for a separation of duties. Testing can be performed through the review of access rules to ensure that access has been granted as management intended. Activity reports provide details, by user, of activity volume and hours. Activity reports should be reviewed to ensure that activity occurs only during authorized hours of operation.

Violation reports indicate any unsuccessful and unauthorized access attempts. Violation reports should indicate the terminal location, date and time of attempted access. These reports should evidence managerial review. Repeated unauthorized access violations may indicate attempts to circumvent access controls. Testing may include review of follow-up procedures.

3.14.4 DATA INTEGRITY TESTING

Data integrity testing is a set of substantive tests that examines accuracy, completeness, consistency and authorization of data presently held in a system. It employs testing similar to that used for input control. Data integrity tests will indicate failures in input or processing controls. Controls for ensuring the integrity of accumulated data in a file can be exercised by regularly checking data in the file. When this checking is done against authorized source documentation, it is common to check only a portion of the file at a time. Since the whole file is regularly checked in cycles, the control technique is often referred to as cyclical checking. Two common types of data integrity tests are relational and referential integrity tests. Relational integrity tests are performed at the data element and record-based levels. Relational integrity is enforced through data validation routines built into the application or by defining the input condition constraints and data characteristics at the table definition in the database stage. Sometimes it is a combination of both. Referential integrity tests define existence relationships between entities in a database that needs to be maintained by the DBMS. It is required for maintaining interrelation integrity in the relational data model. Whenever two or more relations are related through referential constraints (primary and foreign key), it is necessary that references be kept consistent in the event of insertions, deletions and updates to these relations. Database software generally provides various built-in automated procedures for checking and ensuring referential integrity. Referential integrity checks involve ensuring that all references to a primary key from another file (i.e., a foreign key) actually exist in their original file. In nonpointer databases (e.g., relational), referential integrity checks involve making sure that all foreign keys exist in their original table.

3.14.5 DATA INTEGRITY IN ONLINE TRANSACTION PROCESSING SYSTEMS

In multiuser transaction systems, it is necessary to manage parallel user access to stored data typically controlled by a DBMS, and deliver fault tolerance. Of particular importance are four online data integrity requirements known collectively as the ACID principle: AtomicityFrom a user perspective, a transaction is either completed in its entirety (i.e., all relevant database tables are updated) or not at all. If an error or interruption occurs, all changes made up to that point are backed out. ConsistencyAll integrity conditions in the database are maintained with each transaction, taking the database from one consistent state into another consistent state. IsolationEach transaction is isolated from other transactions and hence each transaction only accesses data that are part of a consistent database state. DurabilityIf a transaction has been reported back to a user as complete, the resulting changes to the database survive subsequent hardware or software failures. This type of testing is vital in today's vast array of online Internet-accessible, multiuser DBMSs.

3.14.6 TEST APPLICATION SYSTEMS

Testing the effectiveness of application controls involves analyzing computer application programs, testing computer application program controls, or selecting and monitoring data process transactions. Testing controls by applying appropriate audit procedures is important to ensure their functionality and effectiveness. Methods and techniques for each category are described in exhibit 3.28.

To facilitate the evaluation of application system tests, an IS auditor may also want to use generalized audit software (GAS). This is particularly useful when specific application control weaknesses are discovered that affect, for example, updates to master file records and certain error conditions on specific transaction records. Additionally, GAS can be used to perform certain application control tests, such as parallel simulation, in comparing expected outcomes to live data.

3.15 AUDITING SYSTEMS DEVELOPMENT, ACQUISITION AND MAINTENANCE

The IS auditor's tasks in system development, acquisition and maintenance generally include the following: Meet with key systems development and user project team members to determine the main components, objectives and user requirements of the system to identify the areas that require controls. Discuss the selection of appropriate controls with systems development and user project team members to determine and rank the major risks to and exposures of the system. Discuss references to authoritative sources with systems development and user project team members to identify controls to mitigate the risks to and exposures of the system. Evaluate available controls and participate in discussions with systems development and user project team members to advise the project team regarding the design of the system and implementation of controls. Periodically meet with systems development and user project team members, and review the documentation and deliverables to monitor the systems development process to ensure that controls are implemented, user and business requirements are met, and the systems development/acquisition methodology is being followed. Also review and evaluate the application system audit trails to ensure that documented controls are in place to address all security, edit and processing controls. Audit trails are tracking mechanisms that can help IS auditors ensure program change accountability. Tracking information in a change management system includes: o History of all work order activity (date of work order, programmer assigned, changes made and date closed) o History of logons and logoffs by programmers o History of program deletions Participate in postimplementation reviews. Review appropriate documentation, discuss with key personnel, and use observation to evaluate system maintenance standards and procedures to ensure their adequacy. Discuss and examine supporting records to test system maintenance procedures to ensure that they are being applied as described in the standards. Analyze test results and other audit evidence to evaluate the system maintenance process to determine whether control objectives were achieved. Identify and test existing controls to determine the adequacy of production library security to ensure the integrity of the production resources.

3.15.1 PROJECT MANAGEMENT

Throughout the project management process the IS auditor should analyze the associated risks and exposures inherent in each phase of the SDLC and ensure that the appropriate control mechanisms are in place to minimize these risks in a cost-effective manner. Caution should be exercised to avoid recommending controls that cost more to administer than the associated risks the controls are designed to minimize.

When reviewing the SDLC process, the IS auditor should obtain documentation from the various phases and attend project team meetings, offering advice to the project team throughout the system development process. The IS auditor should also assess the project team's ability to produce key deliverables by the promised dates. Typically, the IS auditor should review the adequacy of the following project management activities: Levels of oversight by project committee/board Risk management methods within the project Issue management Cost management Processes for planning and dependency management Reporting processes to senior management Change control processes Stakeholder management involvement Sign-off processAt a minimum, signed approvals from systems development and user management responsible for the cost of the project and/or use of the system Additionally, adequate and complete documentation of all phases of the SDLC process should be evident. Typical types of documentation may include, but should not be limited to, the following: Objectives defining what is to be accomplished during that phase Key deliverables by phases with project personnel assigned direct responsibilities for these deliverables A project schedule with highlighted dates for the completion of key deliverables An economic forecast for that phase, defining resources and the cost of the resources required to complete the phase

3.15.2 FEASIBILITY STUDY

The IS auditor should perform the following functions: Review the documentation produced in this phase for reasonableness. Determine whether all cost justifications/benefits are verifiable and present them, showing the anticipated benefits to be realized. Identify and determine the criticality of the need. Determine if a solution can be achieved with systems already in place. If not, review the evaluation of alternative solutions for reasonableness. Determine the reasonableness of the chosen solution.

3.15.3 REQUIREMENTS DEFINITION

The IS auditor should perform the following functions: Obtain the detailed requirements definition document and verify its accuracy through interviews with the relevant user departments. Identify the key team members on the project team and verify all affected user groups have appropriate representation. Verify that project initiation and cost have received proper management approval. Review the conceptual design specifications (transforms [e.g., Laplace transform], data descriptions) to ensure that they address the needs of the user. Review the conceptual design to ensure that control specifications have been defined. Determine whether a reasonable number of vendors received a proposal covering the project scope and user requirements. Review the UAT specification.

Determine whether the application is a candidate for the use of an embedded audit routine. If so, request that the routine be incorporated in the conceptual design of the system.

PRACTICE 3-6 When auditing the requirements phase of a software acquisition, the IS auditor QUESTION should: A. assess the feasibility of the project timetable. B. assess the vendor's proposed quality processes. C. ensure that the best software package is acquired. D. review the completeness of the specifications. Answers

3-6 D The purpose of the requirements phase is to specify the functionality of the proposed system; therefore, the IS auditor would concentrate on the completeness of the specifications. The decision to purchase a package from a vendor would come after the requirements have been completed. Therefore, choices B and C are incorrect. Choice A is incorrect because a project timetable normally would not be found in a requirements document.

3.15.4 SOFTWARE ACQUISITION PROCESS

The IS auditor should perform the following functions: Analyze the documentation from the feasibility study to determine whether the decision to acquire a solution was appropriate. Review the RFP to ensure that it covers the items listed in this section. Determine whether the selected vendor is supported by RFP documentation. Attend agenda-based presentations and conference room pilots to ensure that the system matches the vendor's response to the RFP. Review the vendor contract prior to its signing to ensure that it includes the items listed. Ensure the contract is reviewed by legal counsel before it is signed. PRACTICE QUESTION 3-7 An organization decides to purchase a software package instead of developing it. In such a case, the design and development phases of a traditional software development life cycle (SDLC) would be replaced with: A. selection and configuration phases. B. feasibility and requirements phases. C. implementation and testing phases. D. nothing; replacement is not required.

Answers

3-7 A With purchase packages becoming more common, the design and development phases of the traditional life cycle have become replaceable with selection and configuration phases. A request for proposal from the supplier of packaged systems is called for and evaluated against predefined criteria for selection, before a decision is made to purchase the software. Thereafter, it is configured to meet the organization's requirement. The other phases of the SDLC such as feasibility study, requirements definition, implementation and postimplementation remain unaltered.

3.15.5 DETAILED DESIGN AND DEVELOPMENT

The IS auditor should perform the following functions: Review the system flowcharts for adherence to the general design. Verify that appropriate approvals were obtained for any changes and all changes were discussed and approved by appropriate user management. Review the input, processing and output controls designed into the system for appropriateness. Interview the key users of the system to determine their understanding of how the system will operate, and assess their level of input into the design of screen formats and output reports. Assess the adequacy of audit trails to provide traceability and accountability of system transactions. Verify the integrity of key calculations and processes. Verify that the system can identify and process erroneous data correctly. Review the quality assurance results of the programs developed during this phase. Verify that all recommended corrections to programming errors were made and the recommended audit trails or EAMs were coded into the appropriate programs. PRACTICE QUESTION 3-8 User specifications for a project using the traditional SDLC methodology have not been met. An IS auditor looking for a cause should look in which of the following areas? A. Quality assurance B. Requirements C. Development D. User training

Answers

3-8 C Obviously, project management has failed to either set up or verify controls that provide for software or software modules under development that adhere to those specifications made by the users. Quality assurance has its focus on formal aspects of software development such as adhering to coding standards or a specific development methodology. Functionality issues in terms of business usability are out of scope and are, in this case, part of the development phase. To fail at user specifications implies that requirements engineering has been done to describe the users' demands. Otherwise, there would not be a baseline of specifications to check against. Depending on the chosen approach (traditional waterfall, RAD, etc.), these discrepancies might show up during user training or acceptance testingwhatever occurs first.

3.15.6 TESTING
Testing is crucial in determining the user requirements have been validated, the system is performing as anticipated and internal controls work as intended. Therefore it is essential that the IS auditor be involved in reviewing this phase and perform the following: Review the test plan for completeness; indicate evidence of user participation such as user development of test scenarios and/or user sign-off of results; and consider rerunning critical tests. Reconcile control totals and converted data. Review error reports for their precision in recognizing erroneous data and resolution of errors. Verify cyclical processing for correctness (month-end, year-end processing, etc.). Interview end users of the system for their understanding of new methods, procedures and operating instructions. Review system and end-user documentation to determine its completeness and verify its accuracy during the test phase. Review parallel testing results for accuracy. Verify that system security is functioning as designed by developing and executing access tests. Review unit and system test plans to determine whether tests for internal controls are planned and performed. Review the user acceptance testing and ensure that the accepted software has been delivered to the implementation team. The vendor should not be able to replace this version. Review procedures used for recording and following through on error reports.

3.15.7 IMPLEMENTATION PHASE

This phase is initiated only after a successful testing phase. The system should be installed according to the organization's change control procedures. The IS auditor should verify that appropriate sign-offs have been obtained prior to implementation and perform the following: Review the programmed procedures used for scheduling and running the system along with system parameters used in executing the production schedule. Review all system documentation to ensure its completeness and that all recent updates from the testing phase have been incorporated. Verify all data conversion to ensure that they are correct and complete before implementing the system in production.

3.15.8 POSTIMPLEMENTATION REVIEW

After the new system has stabilized in the production environment, a postimplementation review should be performed. Prior to this review, it is important that sufficient time be allowed for the system to stabilize in production. In this way, any significant problems will have had a chance to surface. The IS auditor should perform the following functions: Determine if the system's objectives and requirements were achieved. During the postimplementation review, careful attention should be paid to the end users' utilization and overall satisfaction with the system. This will indicate whether the system's objectives and requirements were achieved. Determine if the cost benefits identified in the feasibility study are being measured, analyzed and accurately reported to management. Review program change requests performed to assess the type of changes required of the system. The type of changes requested may indicate problems in the design, programming or interpretation of user requirements. Review controls built into the system to ensure that they are operating according to design. If an EAM was included in the system, use this module to test key operations. Review operators' error logs to determine if there are any resource or operating problems inherent within the system. The logs may indicate inappropriate planning or testing of the system prior to implementation. Review input and output control balances and reports to verify that the system is processing data accurately.

3.15.9 SYSTEM CHANGE PROCEDURES AND THE PROGRAM MIGRATION PROCESS

Following implementation and stabilization, a system enters into the ongoing development or maintenance stage. This phase continues until the system is retired. The phase involves those activities required to either correct errors in the system or enhance the capabilities of the system. In this regard, the IS auditor should consider the following: The use and existence of a methodology for authorizing, prioritizing and tracking system change requests from the user Whether emergency change procedures are addressed in the operations manuals Whether change control is a formal procedure for the user and the development groups Whether the change control log ensures all changes shown were resolved The user's satisfaction with the turnaroundtimeliness and costof change requests The adequacy of the security access restrictions over production source and executable modules The adequacy of the organization's procedures for dealing with emergency program changes The adequacy of the security access restrictions over the use of the emergency logon IDs For a selection of changes on the change control log: Determine whether changes to requirements resulted in appropriate change-development documents such as program and operations documents. Determine whether changes were made as documented. Determine whether current documentation reflects the changed environment. Evaluate the adequacy of the procedures in place for testing system changes. Review evidence (test plans and test results) to ensure that procedures are carried out as prescribed by organizational standards. Review the procedures established for ensuring executable and source code integrity. Review production executable modules and verify there is one and only one corresponding version of the program source code.

Additionally, the IS auditor should review the overall change management process for possible improvements in acknowledgement, response time, response effectiveness and user satisfaction with the process.