CASELET: JAYHAWK MEDICAL SUPPLY INC.

2. CASELET: JAYHAWK MEDICAL SUPPLY INC.

LEARNING OBJECTIVE
Design an audit program for testing the operating effectiveness of internal controls over changes to an information system.

CASE INFORMATION
Jayhawk Medical Supply Inc. (JMS) sells a comprehensive line of durable equipment and disposable supplies to hospitals, clinics and
doctors offices. Its product line ranges from tongue depressors and cotton swabs to x-ray machines and magnetic resonance imaging
equipment with six-figure price tags. Customer service is provided by salespeople assigned to specific geographic areas, but customers
are encouraged to place routine orders through the JMS web site. Inventory is maintained in six regional warehouses. Products are
delivered to customers by common carriers so that the company does not have to manage and maintain a fleet of delivery vehicles.
The JMS business strategy relies heavily on information technology (IT) to provide fast and efficient customer service, and to minimise
supply chain costs. Its information system was assembled through a best-of-breed strategy that involved buying from different vendors:
Database management
Customer relationship management
Supply chain management
Accounting/human resource
Business intelligence applications
These systems are integrated by a middleware package purchased from and maintained by a sixth vendor. This configuration provides
JMS with an enterprise system that is custom-tailored to its business needs, but also requires constant changes from installing upgrades
to maintain the functionality of the six major application packages.
Because IT is essential to the customer value proposition that supports the JMS business strategy, the enterprise system is maintained by
an in-house IT organisation, which is directed by a chief information officer (CIO) who reports directly to the chief executive officer
(CEO). JMS offers a wide range of sales and payment options to its customers, and management has entered into a variety of
arrangements for procurement and vendor-managed inventory. Therefore, the enterprise system also plays an important role in recognising
or deferring sales revenue, and in determining when costs should be capitalised as inventory or expensed as cost of goods sold.
Under the guidance of the IT investment committee of the board of directors, the IT organisation acquires all hardware and software
solutions (including system upgrades) on a turnkey basis from well-recognised IT vendors. No programming or system development
activities are conducted in-house. With the help of vendor personnel, the IT organisation has installed system upgrades for four of the
five major applications during the past year, and modified the middleware application to accommodate these changes. JMS has designed
a system of internal controls that management attempts to enforce for all changes to the enterprise system.

CASE QUESTION
Assume that you are directing field work for a team of auditors who are evaluating the effectiveness of internal controls over system
changes. The audit team has already evaluated the design effectiveness of these controls and concluded that JMS has adequate
procedures in place. The team has identified critical system change controls that need to be tested to provide evidence for evaluating the
operating effectiveness. Your task is to design a program of audit tests. Your firms inventory of potential audit procedures for controls
over system changes is presented as an appendix to this case. For each of the critical controls listed below, describe specific test
procedures that should be used to evaluate the operating effectiveness of the control practice. Focus on designing the appropriate test
procedure; you do not have to specify sample sizes or timing for the procedures you describe.

Critical Controls
1. AI6.1 Set up formal change management procedures to handle in a standardised manner all requests (including maintenance and
patches) for changes to applications, procedures, processes, system and service parameters, and the underlying platforms.
2. AI6.2 Assess all requests for change in a structured way to determine the impact on the operational system and its functionality.
Ensure that changes are categorised, prioritised and authorised.
3. AI6.3 Establish a process for defining, raising, testing, documenting, assessing and authorising emergency changes that do not follow
the established change process.
IT GOVERNANCE INSTITUTE

IT GOVERNANCE USING COBIT AND VAL IT

TM

CASELETS, 2 EDITION
ND

4. AI6.4 Establish a tracking and reporting system to document rejected changes, the status of approved and in-process changes, and
complete changes. Make certain that approved changes are implemented as planned.
5. AI6.5 Whenever changes are implemented, update the associated system and user documentation and procedures accordingly.
6. AI7.1 Train the staff members of the affected user departments and the operations group of the IT function in accordance with the
defined training and implementation plan and associated materials, as part of every information systems development,
implementation or modification project.
7. AI7.5 Plan data conversion and infrastructure migration as part of the organisations development methods, including audit trails,
rollbacks and fallbacks.
8. AI7.7 Ensure that business process owners and IT stakeholders evaluate the outcome of the testing process as determined by the test
plan. Remediate significant errors identified in the testing process, having completed the suite of tests identified in the test plan and
any necessary regression tests. Following evaluation, approve promotion to production.
9. AI7.9 Establish procedures in line with the organisational change management standards to require a post-implementation review as
set out in the implementation plan.
10. DS5.3 Ensure that all users (internal, external and temporary) and their activity on IT systems (business application, IT environment,
system operations, development and maintenance) are uniquely identifiable. Enable user identities via authentication mechanisms.
Confirm that user access rights to systems and data are in line with defined and documented business needs, and that job
requirements are attached to user identities. Ensure that user access rights are requested by user management, approved by system
owners and implemented by the security-responsible person. Maintain user identities and access rights in a central repository. Deploy
cost-effective technical and procedural measures, and keep them current to establish user identification, implement authentication
and enforce access rights.

CONTROL OBJECTIVES AND PROCEDURES FOR EVALUATING EFFECTIVENESS

AI6.1 Change Standards and ProceduresSet up formal change management procedures to handle in a standardised manner all requests (including
maintenance and patches) for changes to applications, procedures, processes, system and service parameters, and the underlying platforms.
Control Practices
Assurance Steps
1

2
3
4

Develop, document and promulgate a change management

framework that specifies the policies and processes,
including:
Roles and responsibilities
Classification and prioritisation of all changes based
on business risk
Assessment of impact
Authorisation and approval of all changes by the
business process owners and IT
Tracking and status of changes
Impact on data integrity (e.g., all changes to data files
made under system and application control rather than
by direct user intervention)
Establish and maintain version control over all changes.
Implement roles and responsibilities that involve business
process owners and appropriate technical IT functions.
Ensure appropriate segregation of duties.
Establish appropriate record management practices and
audit trails to record key steps in the change management
process. Ensure timely closure of changes. Elevate and
report to management changes that are not closed in a
timely fashion.
Consider the impact of contracted service providers
(e.g., of infrastructure, application development and
shared services) on the change management process.
Consider integration of organisational change
management processes with change management
processes of service providers. Consider the impact of the
organisational change management process on contractual
terms and service level agreements (SLAs).

Enquire whether and confirm that the processes and

procedures for handling change requests (including
maintenance and patches) apply to applications, procedures,
processes, system and service parameters, and the underlying
platforms.
Review the change management framework to determine if the
framework includes:
The definition of roles and responsibilities
Classification (e.g., between infrastructure and application
software) and prioritisation of all changes
Assessment of impact, authorisation and approval
Tracking of changes
Version control mechanism
Impact on data integrity (e.g., all changes to data files made
under system and application control rather than by direct user
intervention)
Management of change from initiation to review and closure
Definition of rollback procedures
Use of emergency change processes
Business continuity planning
Use of a record management system
Audit trails
Segregation of duties
Enquire whether and confirm that processes and procedures for
contracted services providers (e.g., infrastructure, application
development, application service providers, shared services) are
included in the change management process.
Determine if the process and procedures include the contractual
terms and SLAs.

IT GOVERNANCE INSTITUTE

CASELET: JAYHAWK MEDICAL SUPPLY INC.

AI6.2 Impact Assessment, Prioritisation and AuthorisationAssess all requests for change in a structured way to determine the impact on the
operational system and its functionality. Ensure that changes are categorised, prioritised and authorised.
Control Practices
Assurance Steps
1 Develop a process to allow business process owners and IT to
Enquire whether and confirm that the change management process allows
request changes to infrastructure, systems or applications.
business process owners and IT to request changes to infrastructure,
Develop controls to ensure that all such changes arise only
systems or applications.
through the change request management process.
Enquire whether and confirm that requested changes are categorised
2 Categorise all requested changes (e.g., infrastructure, operating
(e.g., amongst infrastructures, operating systems, networks, application
systems, networks, application systems, purchased/packaged
systems, purchased/packaged application software).
application software).
Confirm through interviews with key staff members that requested
3 Prioritise all requested changes. Ensure that the change
changes are prioritised based on predefined criteria (e.g., business and
management process identifies both the business and technical
technical needs for the change and legal, regulatory and contractual
needs for the change. Consider legal, regulatory and contractual
requirements).
reasons for the requested change.
Enquire whether and confirm that change requests are assessed and
4 Assess all requests in a structured fashion. Ensure that the
documented in a structured method that addresses impact analysis on
assessment process addresses impact analysis on infrastructure, infrastructure, systems and applications.
systems and applications. Consider security, legal, contractual
Enquire whether and confirm that security, legal, contractual and
and compliance implications of the requested change. Consider
compliance implications are considered in the assessment process for
also interdependencies amongst changes. Involve business
the requested change and that business owners are involved.
process owners in assessment process as appropriate.
Enquire whether and confirm that each requested change is formally
5 Ensure that each change is formally approved by business
approved by the business process owners and IT technical stakeholders.
process owners and IT technical stakeholders, as appropriate.
Inspect a representative sample of change management requests to
ensure that they were appropriately assessed, evaluated, prioritised
and reviewed.

AI6.3 Emergency ChangesEstablish a process for defining, raising, testing, documenting, assessing and authorising emergency changes that do not
follow the established change process.
Control Practices
Assurance Steps
1 Ensure that a documented process exists within the overall
Enquire whether and confirm that the overall change management process
change management process to declare, assess, authorise and
includes emergency change procedures (e.g., defining, raising, testing,
record an emergency change.
documenting, assessing and authorising emergency changes).
2 Ensure that emergency changes are processed in accordance with Inspect the documentation for a representative sample of emergency
the emergency change element of the formal change
changes and, by interviewing key staff members, establish whether
management process.
emergency changes are implemented as specified in the change
3 Ensure that all emergency access arrangements for changes are
management process.
appropriately authorised, documented and revoked after the
Confirm through interviews with key staff members that emergency access
change has been applied.
arrangements are authorised, documented and revoked after the change
4 Conduct a post-implementation review of all emergency changes,
has been applied.
involving all concerned parties. The review should consider
Enquire whether and confirm that a post-implementation review of
implications for aspects such as further application system
emergency changes is conducted.
maintenance, impact on development and test environments,
application software development quality, documentation and
manuals, and data integrity.
AI6.4 Change Status Tracking and ReportingEstablish a tracking and reporting system to document rejected changes, the status of approved and inprocess changes, and complete changes. Make certain that approved changes are implemented as planned.
Control Practices
Assurance Steps
1 Establish a process to allow requestors and stakeholders to track Enquire whether and confirm that there is an established process to allow
the status of requests throughout the various stages of the change requestors and stakeholders to track the status of requests throughout the
management process.
various stages of the change management process.
2 Categorise change requests in the tracking process (e.g. as
Enquire whether and confirm that the tracking and reporting system
rejected, approved but not yet initiated, approved and in-process,
monitors the status of the change requests (e.g., rejected, approved but not
and closed).
initiated, approved and in process).
3 Implement change status reports with performance metrics to
Enquire whether and confirm that management reviews and monitors the
enable management review and monitoring of both the detailed
detailed status of changes and overall state (e.g., aged analysis of change
status of changes and the overall state (e.g., aged analysis of
requests).
change requests). Ensure that status reports form an audit trail so Enquire whether and confirm that open and approved changes are closed in
changes can subsequently be tracked from inception to eventual
a timely manner, depending on priority.
disposition.
4 Monitor open changes to ensure that all approved changes are
closed in a timely fashion, depending on priority.

IT GOVERNANCE INSTITUTE

IT GOVERNANCE USING COBIT AND VAL IT

TM

CASELETS, 2 EDITION
ND

AI6.5 Change Closure and DocumentationWhenever changes are implemented, update the associated system and user documentation and procedures
accordingly.
Control Practices
Assurance Steps
1 Ensure that documentationincluding operational procedures,
Enquire whether and confirm that change documentation (e.g., operational
configuration information, application documentation, help
procedures, configuration information, application documentation, help
screens and training materialsfollows the same change
screens, training materials) is up to date.
management procedure and is considered to be an integral part
Enquire whether and confirm that change documentation (e.g., pre- and
of the change.
post-implementation system and user documentation) is retained.
2 Consider an appropriate retention period for change
Enquire whether and confirm that business process documentation is
documentation and pre- and post-change system and user
updated for the changes implemented in hardware or software.
documentation.
3 Update business processes for changes in hardware or software
to ensure that new or improved functionality is used.
4 Subject documentation to the same level of testing as the
actual change.
AI7.1 TrainingTrain the staff members of the affected user departments and the operations group of the IT function in accordance with the defined
training and implementation plan and associated materials, as part of every information systems development, implementation or modification project.
Control Practices
Assurance Steps
1

5
6

For systems development, implementation or modification

projects, a training plan is an integral part of the overall project
master plan. Ensure that the plan clearly identifies learning
objectives, resources, key milestones, dependencies and critical
path tasks impacting the delivery of the training plan. The plan
should consider alternative training strategies depending on the
business needs, risk level (e.g., for mission-critical systems, a
formal system of user accreditation and reaccreditation may be
appropriate), and regulatory and compliance requirements
(e.g., impact of varying privacy laws may require adaptation of the
training at a national level).
Ensure that the training plan identifies and addresses all impacted
groups, including business end users, IT operations, support and
IT application development training, and service providers. The
training plan should incorporate the delivery of the training in a
timely manner. It should also identify staff members who must be
trained and those for whom training is desirable.
Consider alternative training strategies that satisfy the training
requirements, and select the most cost-effective approach that
aligns with the organisations training framework. Alternative
strategies include training the trainer, end-user accreditation and
intranet-based training.
Confirm that there is a process to ensure that the training plan is
executed satisfactorily. Complete the documentation detailing
compliance with the training plan. Examples of information include
lists of staff members invited to attend the training, attendees,
evaluations of achievement of learning objectives and other
feedback.
Monitor training to obtain feedback that could lead to potential
improvements in either the training or the system.
Monitor all planned changes to ensure that training requirements
have been considered and suitable plans created. Consider
postponing the change if training has not been performed and the
lack of training would jeopardise the implementation of the change.

Enquire whether and confirm that a training plan is part of the overall project
master plan for development projects.
Enquire whether and confirm (e.g., through interviews with key staff
members or inspection of project plan) that the training plan identifies and
addresses impacted groups (e.g., business end users, IT operations,
support and IT application development training, service providers).
Enquire whether and confirm that alternative training strategies are
considered to ensure that a cost-effective approach is selected and
incorporated in the training framework.
Enquire whether and confirm that there is a process to verify compliance
with the training plan.
Inspect training documentation to determine compliance with the training
plan (e.g., list of staff members invited to training, attendees list, evaluation
forms for the achievement of learning objectives, other feedback).
Enquire whether and confirm that there is a process of monitoring training
to obtain feedback that could lead to potential improvements in the system.
Enquire whether and confirm that planned changes are monitored to ensure
that training requirements are considered and suitable plans are created.

IT GOVERNANCE INSTITUTE

CASELET: JAYHAWK MEDICAL SUPPLY INC.

AI7.5 System and Data ConversionPlan data conversion and infrastructure migration as part of the organisations development methods, including
audit trails, rollbacks and fallbacks.
Control Practices
Assurance Steps
1

Define a data conversion and infrastructure migration plan.

Confirm (e.g., through interviews with key staff members or inspection of
Consider, for example, hardware, networks, operating systems,
policies and procedures) that data conversion and infrastructure mitigation
software, transaction data, master files, backups and archives,
plans exist, and consider the following: hardware, networks, operating
interfaces with other systems (both internal and external),
systems, software, transaction data, master files, backups and archives,
procedures, and system documentation in the development of
interfaces with other internal and external systems, procedures, system
the plan.
documentation, etc.
Ensure that the data conversion plan incorporates methods for
Through interviews with key staff members, enquire about the timing and
collecting, converting and verifying data to be converted, and
completeness of conversion cutover.
identifying and resolving any errors found during conversion. This Enquire whether and confirm that a backup is taken prior to conversion,
includes comparing the original and converted data for
audit trails are maintained, and a fallback and recovery plan exists.
completeness and integrity.
Confirm that the data conversion plan does not require changes in
data values unless absolutely necessary for business reasons.
Document changes made to data values, and secure approval
from the business process data owner.
Consider real-time disaster recovery, business continuity planning,
and reversion in the data conversion and infrastructure migration
plan where risk management, business needs, or regulatory or
compliance requirements demand.
Co-ordinate and verify the timing and completeness of conversion
cutover so there is a smooth, continuous transition with no loss
of transactions. Where necessary, in the absence of any other
alternative, freeze live operations.
Ensure that there is a backup of all system and data taken at the
point prior to conversion, audit trails are maintained to enable the
conversion to be retraced, and there is a fallback and recovery
plan in case the conversion fails. Ensure that the retention of
backup and archived data conforms to business needs and
regulatory or compliance requirements.

AI7.7 Final Acceptance TestEnsure that business process owners and IT stakeholders evaluate the outcome of the testing process as determined by
the test plan. Remediate significant errors identified in the testing process, having completed the suite of tests identified in the test plan and any
necessary regression tests. Following evaluation, approve promotion to production.
Control Practices
Assurance Steps
1

3
4
5

Ensure that the scope of final acceptance evaluation activities

covers all components of the information system (e.g., application
software, facilities, technology, user procedures, operations
procedures, monitoring and support).
Ensure that the categorised log of errors found in the testing
process has been addressed by the development team. Ensure
that the cause of errors has been remediated (e.g., by appropriate
changes to the application or configuration or workaround and/or
delayed correction where the error is minor).
Ensure that the final acceptance evaluation is measured against
the success criteria set out in the testing plan. Ensure that the
review and evaluation process is appropriately documented.
Document and interpret the final acceptance testing results, and
present them in a form that is understandable to business process
owners and IT so an informed review and evaluation can take place.
Ensure that business process owners, appropriate third parties
and IT stakeholders formally sign off on the outcome of the
testing process as set out in the testing plan. Such approval is
mandatory prior to promotion to production.

Confirm that key stakeholders are considered in the final acceptance testing
activities.
Enquire whether and confirm that success criteria are identified in the
testing plan in the final acceptance stages.
Enquire whether and confirm that appropriate documentation for review and
evaluation exists.
Enquire of key stakeholder whether the documentation and presentation of
final acceptance testing results are complete and timely.

IT GOVERNANCE INSTITUTE

IT GOVERNANCE USING COBIT AND VAL IT

TM

CASELETS, 2 EDITION
ND

AI7.9 Post-implementation ReviewEstablish procedures in line with the organisational change management standards to require a post-implementation
review as set out in the implementation plan.
Control Practices
Assurance Steps
1

2
3
4
5

Establish procedures to ensure that post-implementation reviews

identify, assess and report on the extent to which:
Business requirements have been met
Expected benefits have been realised
The system is considered useable
Internal and external stakeholders expectations are met
Unexpected impacts on the organisation have occurred
Key risks are mitigated
The change management, installation and accreditation
processes were performed effectively and efficiently
Consult business process owners and IT technical management in
the choice of metrics for measurement of success and
achievement of requirements and benefits.
Ensure that the form of the post-implementation review is in
accordance with the organisational change management process.
Involve business process owners and third parties, as appropriate.
Consider requirements for post-implementation review arising
from outside business and IT (e.g. internal audit, enterprise risk
management, and regulatory and compliance).
Agree on and implement an action plan to address issues
identified in the post-implementation review. Involve business
process owners and IT technical management in the development
of the action plan.

Confirm through interviews with key staff members that post-implementation

procedures have been established.
Confirm through interviews with key staff members that business process
owners and IT technical management are involved in the selection of
metrics for measuring success and achievement of requirements and benefits.
Confirm through interviews with key staff members that the form of the
post-implementation review is in accordance with the organisational change
management process and that business process owners and third parties
are involved, as appropriate.
Confirm through interviews with key staff members that requirements for
post-implementation review arising from outside business and IT are
considered.
Confirm through interviews with key staff members that an action plan
exists to address issues identified in the post-implementation review and
that business process owners and IT technical management are involved in
the development of the action plan.

DS5.3 Identity ManagementEnsure that all users (internal, external and temporary) and their activity on IT systems (business application, IT
environment, system operations, development and maintenance) are uniquely identifiable. Enable user identities via authentication mechanisms. Confirm
that user access rights to systems and data are in line with defined and documented business needs, and that job requirements are attached to user
identities. Ensure that user access rights are requested by user management, approved by system owners and implemented by the security-responsible
person. Maintain user identities and access rights in a central repository. Deploy cost-effective technical and procedural measures, and keep them current
to establish user identification, implement authentication and enforce access rights.
Control Practices
Assurance Steps
1 Establish and communicate policies and procedures to uniquely
Determine if security practices require users and system processes to be
identify, authenticate and authorise access mechanisms and
uniquely identifiable and systems to be configured to enforce authentication
access rights for all users on a need-to-know/need-to-have basis,
before access is granted.
based on predetermined and preapproved roles. Clearly state
If predetermined and preapproved roles are utilised to grant access,
accountability of any user for any action on any of the systems
determine if the roles clearly delineate responsibilities based on least
and/or applications involved.
privileges, and ensure that the establishment and modification of roles are
2 Ensure that roles and access authorisation criteria for assigning
approved by process owner management.
user access rights take into account:
Determine if access provisioning and authentication control mechanisms
Sensitivity of information and applications involved
are utilised for controlling logical access across all users, system processes
(data classification)
and IT resources, for in-house and remotely managed users, processes
Policies for information protection and dissemination (legal,
and systems.
regulatory, internal policies and contractual requirements)
Roles and responsibilities as defined within the enterprise
The need-to-have access rights associated with the function
Standard but individual user access profiles for common job
roles in the organisation
Requirements to guarantee appropriate segregation of duties
3 Establish a method for authenticating and authorising users to
establish responsibility and enforce access rights in line with
sensitivity of information and functional application requirements
and infrastructure components, and in compliance with applicable
laws, regulations, internal policies and contractual agreements.
4 Define and implement a procedure for identifying new users and
recording, approving and maintaining access rights. This needs to
be requested by user management, approved by the system owner
and implemented by the responsible security person.
5 Ensure that a timely information flow is in place that reports
changes in jobs (i.e., people in, people out, people change). Grant,
revoke and adapt user access rights in co-ordination with human
resources and user departments for users who are new, who have
left the organisation, or who have changed roles or jobs.

IT GOVERNANCE INSTITUTE