Professional Documents
Culture Documents
DEVELOPMENT
GUIDE
CISA ITEM DEVELOPMENT GUIDE
TABLE OF CONTENTS
Content Page
Multiple-Choice Items 4
Developing an Item 4
Item Examples 7
2
CISA ITEM DEVELOPMENT GUIDE
As you read through this Guide, please pay particular interest to the item writing
principles appearing on page 6. Applying these principles will greatly enhance the
chances of your items being accepted.
3
CISA ITEM DEVELOPMENT GUIDE
more proficient staff members or managers and would seek assistance on complex
technology issues from technical experts.
While writing items, one must take into consideration that IS audit and control is a global
profession, and individual perceptions and experiences may not reflect the more global
position or circumstance. Since the examination and CISA items are developed for the
international IS audit and control community, this will require the item writer to be
somewhat flexible when determining a globally accepted practice.
MULTIPLE-CHOICE ITEMS
The CISA exam consists of multiple-choice items. The multiple-choice item is the most
commonly used type of test question in certification exams.
Item Stem
The item stem is the introductory statement or question that describes
a situation or circumstance related to the knowledge being assessed. Item
stems can be written in the form of an incomplete statement as well as in
question form.
Item Options
The options complete the introductory statement or answer the
question and consist of one correct answer (key) and three incorrect
answers or distracters.
Key
The key must reflect current practice. In some cases the
key will be the only correct choice, while in other cases the key
will be deemed to be the BEST choice when considered with the
other choices provided.
Distracters
Distracters are the incorrect options but should be plausible or
possible correct answers to candidates who are not knowledgeable
enough to choose the key.
DEVELOPING AN ITEM
Item writers must use the Item Development Form included in Appendix B when
developing and submitting new items to ISACA. Become familiar with this form prior to
item development and make every attempt to complete all sections of the form for each
item submitted.
Item writers can write items on any topic within the CISA Job Practice but are
encouraged to concentrate the development of items within the areas specified in the
CISA item writing campaign. For best results, items should be written on a knowledge
4
CISA ITEM DEVELOPMENT GUIDE
statement within a specific task area. Items should focus on a single topic or knowledge
statement.
Once a topic is chosen, follow the steps listed below. While writing your item, please
refer to the General Item Writing Principles appearing on page 6 for further guidance and
review your item using the Item Development Checklist found in Appendix C.
STEP 1 – Write the item stem and keyable answer (Answer A on the Item Development
Form).
STEP 2 – Develop plausible distracters. The distracters should not be made up words or
phrases, but may appear to be correct choices to an inexperienced professional. The
development of quality distracters is usually the most difficult task for an item writer. If
you have difficulty with this part of item development, consult with other staff members
or subject matter experts.
STEP 3 - Include a thorough explanation of why the keyable answer is correct as well as
why each distracter is not a correct choice. It is not acceptable to simply state that the
distracters are incorrect.
STEP 4 - If reference texts are used for the development of an item the reference sources
should be noted on the Item Development Form. Refer to KNET on the ISACA website
for applicable references – http://www.isaca.org/knet
STEP 5 – Review your item using the Item Development Checklist found in Appendix C.
STEP 7 – Submit the item to ISACA. (Refer to Item Submission and Review Process on
page 9).
5
CISA ITEM DEVELOPMENT GUIDE
DO:
1. DO test only one testing concept or knowledge statement per item. Knowledge
statements were developed for this purpose and items written from a knowledge
statement will most likely result in higher quality, practically based items. For a listing of
knowledge statements, refer to Appendix A “CISA Job Practice Analysis” or
http://www.isaca.org/cisacontentareas
2. DO ensure that the stem and all options are compatible with each other. For example,
if your stem reads, “Which of the following audit procedures… ”, then all options must
be audit procedures.
3. DO keep the stem and options as short as possible by avoiding the use of unnecessary
text or jargon. Do not attempt to teach the candidate a concept or theory by providing too
much information before asking the question.
4. DO include common words or phrases in the item stem rather than in the key and
distracters.
5. DO write all options the same approximate length and format.
6. DO write options that are grammatically consistent with the item stem and maintain a
parallel grammatical format. For example if the key begins with a verb ending with ing,
then all distracters must begin with a verb ending with ing.
7. DO use only professionally acceptable or technical terminology in the item stem and
options. If in doubt, refer to the CISA terminology list for acceptable terminology
http://www.isaca.org/examterm
DO NOT:
1. DO NOT use a key word or phrase in the item key that appears in the stem.
Experienced test takers will look for clues such as this that often identify the key.
2. DO NOT use words such as “frequently”, “often”, “common”, or “rarely” as they
introduce subjectivity into the item. If an item is subjective, it can be argued that more
than one option is keyable.
3. DO NOT use terms in the stem such as “always”, “never”, or “all” since very little is
absolute and thus it makes it easier for candidates to eliminate distracters.
4. DO NOT use terms such as “least”, “not” or “except” as they are negative and require
a candidate to choose an incorrect or least preferred choice, rather than a correct or
preferred choice. These questions will be returned to the item writer without TEC
review.
5. DO NOT use gender pronouns such as he, she, his, or her. Refer to individuals by their
title, such as the IS auditor.
6. DO NOT use “all of the above”, “none of the above”, as options. These questions will
be returned to the item writer without TEC review.
6
CISA ITEM DEVELOPMENT GUIDE
7. DO NOT test knowledge regarding vendor specific products. These questions will be
returned to the item writer without TEC review.
ITEM EXAMPLES
Direct question
Stem: Which of the following concerns would BEST be addressed by the comparison of
production application systems source code with an archive copy?
Options:
A. File maintenance errors
B. Unauthorized modifications
C. Software version currency
D. Documentation discrepancies
Incomplete statement
Stem: The comparison of production application systems source code with an archive
copy would
BEST address:
Options:
A. file maintenance errors.
B. unauthorized modifications.
C. software version currency.
D. documentation discrepancies.
Note that the responses for this item are followed by a period, as the response serves to
complete the sentence started in the stem.
It is wise to draft an item first as a direct question, and then revise it to an incomplete
sentence if this offers a smoother, less repetitive wording.
Scenario Questions
When writing this type of item, there are a number of considerations that must be kept in
mind. This type of item consists of introductory information or the scenario for the items
to follow. There should be a set of two to five items that pertain to this introductory
7
CISA ITEM DEVELOPMENT GUIDE
correct conclusions – do not force the candidate to make assumptions. The associated
items should be in some sort of sequence and follow a logical progression. Also, each
item should be independent of the other items so that missing one item does not cause
missing another item of the set. Care should be taken to ensure that one item does not
point to the key of another item.
Example 1
Options:
A. Diverse routing
B. Alternative routing
C. Redundancy
D. Long haul network diversity
Key: A
Notice that a key word from the stem “routing” is in the answer. Avoid using important
words in the stem and the answer or any of the distracters.
Example 2
Options:
A. Functional access controls
B. Logging of changes to loan information
C. Senior management supervision
D. Change management controls
Key: A
8
CISA ITEM DEVELOPMENT GUIDE
The stem assumes functional responsibility. The CISA test is global and it is difficult to
define functional responsibilities between countries and organizations. In some
organizations, the loan department manager may have access.
Example 3
Stem: Spreadsheets are used to calculate project cost estimates. Totals for each cost
category are then keyed into the job costing system. What is the BEST control to ensure
that data entered into the job costing system is accurate?
Options:
A. Reconciliation of total amounts by project.
B. Reasonableness of total amounts by project.
C. Validity checks, preventing entry of character data.
D. Display back of project detail after entry
Key: A
Can a non-IS auditor answer this question? Does this question test an IS audit concept?
There are some questions that IS auditors need to be tested on. This is a borderline
concept. For the most part, we encourage strictly IS audit concepts.
All items must be submitted on an Item Development Form (see Appendix B) and must
be written in English. The Form includes:
9
CISA ITEM DEVELOPMENT GUIDE
Each item submitted must include all fields listed on the Item Development Form. Please
use Microsoft Word when submitting items. Multiple items should be included in one
document. All items MUST be submitted in English.
Submit completed forms to ISACA for initial review. These can be e-mailed to
cisa@isaca.org, or mailed to the Exam Development Coordinator at ISACA headquarters,
3701 Algonquin Road, Suite 1010, Rolling Meadows, Illinois 60008 USA.
Items that pass the initial review will be forwarded for review and critique by the full
membership of the CISA TEC. At this point your item(s) may again be accepted or
returned for further work. If returned, the item will include appropriate and constructive
feedback. If accepted, the item will become the property of ISACA and you will receive
the appropriate payment and continuing credit hours for each item accepted.
10
CISA ITEM DEVELOPMENT GUIDE
11
CISA ITEM DEVELOPMENT GUIDE
Appendix A
Task Statements
1.1 Develop and implement a risk-based IS audit strategy for the organization in
compliance with IS audit standards, guidelines and best practices.
1.2 Plan specific audits to ensure that IT and business systems are protected and
controlled.
1.3 Conduct audits in accordance with IS audit standards, guidelines and best practices to
meet planned audit objectives.
1.4 Communicate emerging issues, potential risks, and audit results to key stakeholders.
1.5 Advise on the implementation of risk management and control practices within the
organization while maintaining independence.
Knowledge Statements
1.1 Knowledge of ISACA IS Auditing Standards, Guidelines and Procedures and Code of
Professional Ethics
1.2 Knowledge of IS auditing practices and techniques
1.3 Knowledge of techniques to gather information and preserve evidence (e.g.,
observation, inquiry, interview, CAATs, electronic media)
1.4 Knowledge of the evidence life cycle (e.g., the collection, protection, chain of
custody)
1.5 Knowledge of control objectives and controls related to IS (e.g., CobiT)
1.6 Knowledge of risk assessment in an audit context
1.7 Knowledge of audit planning and management techniques
1.8 Knowledge of reporting and communication techniques (e.g., facilitation, negotiation,
conflict resolution)
1.9 Knowledge of control self-assessment (CSA)
1.10 Knowledge of continuous audit techniques
12
CISA ITEM DEVELOPMENT GUIDE
Content Area 2: IT Governance – To provide assurance that the organization has the
structure, policies, accountability, mechanisms, and monitoring practices in place to
achieve the requirements of corporate governance of IT.
Task Statements
2.1 Evaluate the effectiveness of IT governance structure to ensure adequate board
control over the decisions, directions, and performance of IT so that it supports the
organization’s strategies and objectives.
2.2 Evaluate IT organizational structure and human resources (personnel) management to
ensure that they support the organization’s strategies and objectives.
2.3 Evaluate the IT strategy and the process for its development, approval,
implementation, and maintenance to ensure that it supports the organization’s strategies
and objectives.
2.4 Evaluate the organization’s IT policies, standards, and procedures; and the processes
for their development, approval, implementation, and maintenance to ensure that they
support the IT strategy and comply with regulatory and legal requirements.
2.5 Evaluate management practices to ensure compliance with the organization’s IT
strategy, policies, standards, and procedures.
2.6 Evaluate IT resource investment, use, and allocation practices to ensure alignment
with the organization’s strategies and objectives.
2.7 Evaluate IT contracting strategies and policies, and contract management practices to
ensure that they support the organization’s strategies and objectives.
2.8 Evaluate risk management practices to ensure that the organization’s IT related risks
are properly managed.
2.9 Evaluate monitoring and assurance practices to ensure that the board and executive
management receive sufficient and timely information about IT performance.
Knowledge Statements
2.1 Knowledge of the purpose of IT strategies, policies, standards and procedures for an
organization and the essential elements of each
2.2 Knowledge of IT governance frameworks
2.3 Knowledge of the processes for the development, implementation and maintenance of
IT strategies, policies, standards and procedures (e.g., protection of information assets,
business continuity and disaster recovery, systems and infrastructure lifecycle
management, IT service delivery and support)
2.4 Knowledge of quality management strategies and policies
2.5 Knowledge of organizational structure, roles and responsibilities related to the use
and management of IT
2.6 Knowledge of generally accepted international IT standards and guidelines
2.7 Knowledge of enterprise IT architecture and its implications for setting long-term
strategic directions
2.8 Knowledge of risk management methodologies and tools
2.9 Knowledge of the use of control frameworks (e.g., CobiT, COSO, ISO 17799)
2.10 Knowledge of the use of maturity and process improvement models (e.g., CMM,
CobiT)
13
CISA ITEM DEVELOPMENT GUIDE
Content Area 3: Systems and Infrastructure Lifecycle – To provide assurance that the
management practices for the development/acquisition, testing, implementation,
maintenance, and disposal of systems and infrastructure will meet the organization’s
objectives.
Task Statements
3.1 Evaluate the business case for the proposed system development/acquisition to ensure
that it meets the organization’s business goals.
3.2 Evaluate the project management framework and project governance practices to
ensure that business objectives are achieved in a cost-effective manner while managing
risks to the organization.
3.3 Perform reviews to ensure that a project is progressing in accordance with project
plans, is adequately supported by documentation and status reporting is accurate.
3.4 Evaluate proposed control mechanisms for systems and/or infrastructure during
specification, development/acquisition, and testing to ensure that they will provide
safeguards and comply with the organization’s policies and other requirements.
3.5 Evaluate the processes by which systems and/or infrastructure are developed/acquired
and tested to ensure that the deliverables meet the organization’s objectives.
3.6 Evaluate the readiness of the system and/or infrastructure for implementation and
migration into production.
3.7 Perform post-implementation review of systems and/or infrastructure to ensure that
they meet the organization’s objectives and are subject to effective internal control.
3.8 Perform periodic reviews of systems and/or infrastructure to ensure that they continue
to meet the organization’s objectives and are subject to effective internal control.
3.9 Evaluate the process by which systems and/or infrastructure are maintained to ensure
the continued support of the organization’s objectives and are subject to effective internal
control.
3.10 Evaluate the process by which systems and/or infrastructure are disposed of to
ensure that they comply with the organization’s policies and procedures.
Knowledge Statements
3.1 Knowledge of benefits management practices, (e.g., feasibility studies, business
cases)
3.2 Knowledge of project governance mechanisms (e.g., steering committee, project
oversight board)
3.3 Knowledge of project management practices, tools, and control frameworks
14
CISA ITEM DEVELOPMENT GUIDE
Content Area 4: IT Service Delivery and Support – To provide assurance that the IT
service management practices will ensure the delivery of the level of services required to
meet the organization’s objectives.
Task Statements
4.1 Evaluate service level management practices to ensure that the level of service from
internal and external service providers is defined and managed.
4.2 Evaluate operations management to ensure that IT support functions effectively meet
business needs.
4.3 Evaluate data administration practices to ensure the integrity and optimization of
databases.
4.4 Evaluate the use of capacity and performance monitoring tools and techniques to
ensure that IT services meet the organization’s objectives.
4.5 Evaluate change, configuration, and release management practices to ensure that
changes made to the organization’s production environment are adequately controlled
and documented.
4.6 Evaluate problem and incident management practices to ensure that incidents,
problems, or errors are recorded, analyzed, and resolved in a timely manner.
15
CISA ITEM DEVELOPMENT GUIDE
Knowledge Statements
4.1 Knowledge of service level management practices
4.2 Knowledge of operations management best practices (e.g., workload scheduling,
network services management, preventive maintenance)
4.3 Knowledge of systems performance monitoring processes, tools, and techniques (e.g.,
network analyzers, system utilization reports, load balancing)
4.4 Knowledge of the functionality of hardware and network components (e.g., routers,
switches, firewalls, peripherals)
4.5 Knowledge of database administration practices
4.6 Knowledge of the functionality of system software including operating systems,
utilities, and database management systems
4.7 Knowledge of capacity planning and monitoring techniques
4.8 Knowledge of processes for managing scheduled and emergency changes to the
production systems and/or infrastructure including change, configuration, release, and
patch management practices
4.9 Knowledge of incident/problem management practices (e.g., help desk, escalation
procedures, tracking)
4.10 Knowledge of software licensing and inventory practices
4.11 Knowledge of system resiliency tools and techniques (e.g., fault tolerant hardware,
elimination of single point of failure, clustering)
Task Statements
5.1 Evaluate the design, implementation, and monitoring of logical access controls to
ensure the confidentiality, integrity, availability and authorized use of information assets.
5.2 Evaluate network infrastructure security to ensure confidentiality, integrity,
availability and authorized use of the network and the information transmitted.
5.3 Evaluate the design, implementation, and monitoring of environmental controls to
prevent or minimize loss.
5.4 Evaluate the design, implementation, and monitoring of physical access controls to
ensure that information assets are adequately safeguarded.
5.5 Evaluate the processes and procedures used to store, retrieve, transport, and dispose
of confidential information assets.
Knowledge Statement
5.1 Knowledge of the techniques for the design, implementation and monitoring of
security (e.g., threat and risk assessment, sensitivity analysis, privacy impact assessment)
16
CISA ITEM DEVELOPMENT GUIDE
5.2 Knowledge of logical access controls for the identification, authentication, and
restriction of users to authorized functions and data (e.g., dynamic passwords,
challenge/response, menus, profiles)
5.3 Knowledge of logical access security architectures (e.g., single sign-on, user
identification strategies, identity management)
5.4 Knowledge of attack methods and techniques (e.g., hacking, spoofing, Trojan horses,
denial of service, spamming)
5.5 Knowledge of processes related to monitoring and responding to security incidents
(e.g., escalation procedures, emergency incident response team)
5.6 Knowledge of network and Internet security devices, protocols, and techniques (e.g.,
SSL, SET, VPN, NAT)
5.7 Knowledge of intrusion detection systems and firewall configuration,
implementation, operation, and maintenance
5.8 Knowledge of encryption algorithm techniques (e.g., AESRSA)
5.9 Knowledge of public key infrastructure (PKI) components (e.g., certification
authorities, registration authorities) and digital signature techniques
5.10 Knowledge of virus detection tools and control techniques
5.11 Knowledge of security testing and assessment tools (e.g., penetration testing,
vulnerability scanning)
5.12 Knowledge of environmental protection practices and devices (e.g., fire suppression,
cooling systems, water sensors)
5.13 Knowledge of physical security systems and practices (e.g., biometrics, access
cards, cipher locks, tokens)
5.14 Knowledge of data classification schemes (e.g., public, confidential, private, and
sensitive data)
5.15 Knowledge of voice communications security (e.g., voice over IP)
5.16 Knowledge of the processes and procedures used to store, retrieve, transport, and
dispose of confidential information assets
5.17 Knowledge of controls and risks associated with the use of portable and wireless
devices (e.g., PDAs, USB devices, Bluetooth devices)
Task Statements
6.1 Evaluate the adequacy of backup and restore provisions to ensure the availability of
information required to resume processing.
6.2 Evaluate the organization’s disaster recovery plan to ensure that it enables the
recovery of IT processing capabilities in the event of a disaster.
6.3 Evaluate the organization’s business continuity plan to ensure its ability to continue
essential business operations during the period of an IT disruption.
17
CISA ITEM DEVELOPMENT GUIDE
Knowledge Statements
6.1 Knowledge of data backup, storage, maintenance, retention and restoration processes,
and practices
6.2 Knowledge of regulatory, legal, contractual, and insurance issues related to business
continuity and disaster recovery
6.3 Knowledge of business impact analysis (BIA)
6.4 Knowledge of the development and maintenance of the business continuity and
disaster recovery plans
6.5 Knowledge of business continuity and disaster recovery testing approaches and
methods
6.6 Knowledge of human resources management practices as related to business
continuity and disaster recovery (e.g., evacuation planning, response teams)
6.7 Knowledge of processes used to invoke the business continuity and disaster recovery
plans
6.8 Knowledge of types of alternate processing sites and methods used to monitor the
contractual agreements (e.g., hot sites, warm sites, cold sites)
18
CISA ITEM DEVELOPMENT GUIDE
Appendix B
Domain/Task: 0302
Knowledge Statement: 0317
Item Stem:
In order for a virus vaccine to be an effective preventive control which of the following
should occur?
Item Options:
A. The virus’s signature must be known to the vaccine.
B. The vaccine should be controlled by IS audit.
C. Viral detection software must be used concurrently.
D. The vaccine’s encryption key must be kept secret.
Key: A
Justification:
A. A vaccine recognizes a virus by means of the virus’s signature. If a signature is
unknown the vaccine will not prevent a virus from entering a system.
B. This is neither preventive nor detective.
C. This is a detective control.
D. This is a security policy.
19
CISA ITEM DEVELOPMENT GUIDE
Appendix C
Before submitting an item, you must be able to answer YES to all of the following
questions.
1. Does the item test an IS audit, control or security concept at the appropriate
experience level of the test candidate (3 – 5 years IS audit)?
2. Does the item test only one IS audit, control or security concept?
4. Is there enough information (scenario) in the stem to allow for only one correct
answer? A candidate must not be able to interpret a distracter as correct based on
assumptions due to a lack of information in the stem!
5. Is there only one answer to your stem for any situation, organization or culture?
Many items are returned because there may be a situation when there is more than
one possible keys based on situations not addressed in the stem.
6. Are the stem and all options compatible with each other? For example: “Which
of the following audit procedures…?” All options must be audit procedures.
7. Does your item have plausible distracters but only one correct answer?
8. Have you avoided having words or phrases in the key that appear in the stem?
9. Have you avoided unnecessary text or jargon from the stem or options?
10. Have you avoided using subjective terms such as “frequently”, “often”,
“common”…. in the stem and options?
11. Have you avoided using absolute terms such as “all”, “never”, “always”… in the
stem and options?
12. Have you avoided asking a negative question – using such terms as “least”, “not”,
“except”…?
20