Professional Documents
Culture Documents
Kelvin J. Ross
Abstract
In this work we present a model-based framework for
security vulnerabilities testing. Security vulnerabilities
are not only related to security functionalities at the application level but are sensitive to implementation details. Thus traditional model-based approaches which
elide implementation details are by themselves inadequate for testing security vulnerabilities. We propose a
framework that retains the advantages of model based
testing that exposes only the necessary details relevant
for vulnerability testing. We define a three-model framework: a model or specification of the key aspects of the
application, a model of the implementation and a model
of the attacker, for automatic test case generation. This
separation allows the test case generation process to
test contexts missed by other model-based approaches.
We also describe the key aspects of our tool that generates the tests.
1. Introduction
Security testing aims to find undiscovered vulnerabilities that can be exploited. These vulnerabilities can
be introduced by the functionality of the system that
is not normally associated with security requirements
of the system. Most of the attacks are a result of improper or unexpected interaction with the system. Scott
and Sharp [1] indicate that applications that have a webserver and a database will always be vulnerable. While
there are a number of vulnerability assessment systems
that work at the network layer [2], there are many types
of exploits that cannot be prevented by network level
firewalls and intrusion detection systems. In order to
discover all possible vulnerabilities, one needs to try all
possible improper behaviour at every interaction point.
This is clearly not feasible. Furthermore, new vulnerabilities are discovered as attackers find novel ways to
subvert the behaviour of the system.
Traditional vulnerability analysis has involved penetration testing where known attacks are tried out in various sequences. The response of the system to such
attacks is observed and any inappropriate behaviour
noted. This process requires the knowledge of both the
desired behaviour as well as certain implementation details which are the source of many vulnerabilities. The
various analyses can be formalised in terms of attack
graphs [3] or more scalable representations [4]. Attack patterns for particular vulnerabilities can also be
expressed as regular expressions over behaviours [5].
A limited form of exhaustive testing, i.e., bounded exhaustive testing [6] can also be used to simulate certain
behaviours of attackers.
Specification based or model based testing [7, 8, 9]
is an approach to automate the process of testing specific aspects of behaviour. The success of model based
testing depends on the quality of the model. Ideally,
the actual behaviour of the system will be equivalent to
the one specified by its model. However, in reality, implementations might miss certain functionality and also
present some additional functionality.
During the development process of a complex system not all the system components are built from
scratch. Recent methodologies encourage the use of libraries of pre-built software components. This allows
developers to meet even more and more time and budget
restrictions on the development process. However, the
use of generic components is not free of problems. A
generic component should meet the requirements of not
one but several under-development applications. Then,
these generic components group a lot of functionalities,
a large number of which probably are not used in some
specific application. These additional functionalities,
of flaws.
In our approach we have three main models; one to
describe the desired behaviour of the system, one of the
behaviour of the various relevant components of the implementation and one for the attacker. A model of the
idealised behaviour is necessary for the testing process
as it describes the desired functionality. The model of
the implementation enables the description of the possible exploits while the model of the attacker indicates
potential vulnerabilities using the possible exploits.
In the next section we outline some of the relevant
techniques in model based testing and security testing.
This is followed by the details of our approach and an
example.
2. Background
In this section we describe the basic concepts that
will serve as a basis for our test case generation approach. We mainly focus on a fault-model-based testing
approach and its implementation using state transition
and program models taken from the design of the SpecExplorer tool and its associated language Spec [13].
While such state based techniques allow one to use
model checking techniques to generate test sequences
[14, 15] we do not use it directly. We use constraint
solving to generate test sequences. This will become
clearer in the latter sections of the paper. To automate
vulnerability testing we also need data. Hence we focus
on model based techniques that also provide the basis
for data generation.
op(x)
/ s2 and s1
op(y)
/ s3 in the
op(y)
/ s3 in the implementation.
model and only s1
The fault-based approach tries to find a transition that
exists on the first model but not on the second. This is
op(x)
/ s2 , so this is used
the case of the transition s1
for testing.
Following this approach, Aichernig and Pari Salas
[27] developed an algorithm for test case generation.
They considered two cases in which a faulty model can
be used for test case generation:
when given the same initial state and the same operation (with the same parameters) in a transition,
the final state is different in the two models; and
s1
Application
Input
Test
Data
Data
Output
Test
Constraint Handler
Cases
Attack Model
Generator
Models
3. Model-based
Testing
Security
Vulnerability
We use three separate, but logically related, models for model-based testing of security vulnerabilities.
The first model is a standard specification model, usually abstract and described as a state transition system.
The second model applies the concept of fault injection
and represents a localised mutation of the first model.
It contains low level implementation details related to
a specific security concern and usually would be described as a program model. The third model represents the intentions of an attacker and works as a test
purpose [22] for our method. While it is possible to
further divide the implementation and attacker models
into separate specific security concerns (e.g., one for authentication, another for disclosure of information) we
do not consider that in this paper. The architecture of
our framework is shown if Figure 1.
The attackers model is used to generate suitable
data which are then combined with the fault based approach linking the behaviour and implementation models. Although we use a constraint solver to generate the
test cases, it is possible to use other approaches such as
model checking to generate the test cases.
When a model does not address a certain concern
the model is incomplete or under-specified with respect
to this specific concern. An incomplete model is too
much permissive, thus allowing threats in some implementations by disregarding certain functionalities. Test
cases derived from an incomplete model will conform
to the model while undesired actions or events take
place, thus evaluated with a pass verdict by an automated test execution tool.
Consider an abstract model with the transiop(x)
/ s2 and s1 op(y) / s2 and its
tions s1
implementation with transitions
s1
op(x)
/ s
2
and
op(y)
PreM
op [v] is true; and
M [v ] is true
Postop
S1
op(i1)
S2
S1a
op(i2)
S3
S1b
(a)
S2
op(i1)
S2a
op(i 2 )
S2b
S1
(b)
S4
op(in)
S3
S1
op(i1)
S2
op(i 2 )
(c)
login(usr,pwd)
!map(usr>pwd)
authenticated
login(usr,pwd)
map(usr>pwd)
!authenticated
4. Illustrative example
In this section we present a practical example of
how test cases can be generated using our framework.
The application under test is web based and requires an
authentication process. We will assume that this process is in the form of a login functionality (login page
or login screen) where the user is requested to inform
its username and password to be authenticated. This
has been implemented using a standard web page (using http) and a database (SQL engine) for actually performing the authentication.
login(usr,pwd)
!sqlmap(usr,pwd)
authenticated
operation: login
pre: usr ([AZaz])( )
post: error && columnname error message &&
update knowledge(columnname)
error
login(wf(usr),pwd)
sqlmap(usr,pwd)
login(!wf(usr),pwd)
!error
login(!wf(usr),pwd)
sqlmap(usr,pwd)
!authenticated
users. The attacker will gain legal access to the application without using the regular mechanisms. This is
related with fair exchange security concern if, for example, a payment is required to be included as a user of
the application. This attack should be enabled as a result
of several previous information disclosure attacks. The
unreachable state not considered in the specification is
the state that represents the attacker with enough knowledge of our applications structure. In general this state
is disregarded because the system cannot control what
the user/attacker learns. However, the states where the
knowledge of the attacker is provided by the application
itself cannot be disregarded.
usr = OR(1 = 1)
authenticated = true
Acknowledgments
The first author is supported by a special scholarship from KJRoss and Associates.
References
[1] D. Scott and R. Sharp, Abstracting application-level
web security, in WWW 02: Proceedings of the eleventh
international conference on World Wide Web. ACM
Press, 2002, pp. 396407.
[2] F. Guo, Y. Yu, and T. Chiueh, Automated and safe
vulnerability assessment, in Annual Computer Security
Applications Conference, 2005. [Online]. Available:
http://www.acsac.org
[3] O. Sheyner, S. Jha, and J. M. Wing, Automated generation and analysis of attack graphs, in Proceedings of the
IEEE Symposium on Security and Privacy, May 2002.
[4] P. Ammann, D. Wijesekera, and S. Kaushik, Scalable
Graph-Based Network Vulnerability Analysis, in Computer and Communication Security. ACM, 2002, pp.
217224.
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
Lecture Notes in Computer Science, M. Broy, B. Jonsson, J.-P. Katoen, M. Leucker, and A. Pretschner, Eds.
Springer, July 2005, vol. 3472, pp. 87111.
J. Derrick and E. Boiten, Testing refinements of state-based formal specifications, Software Testing, Verification and Reliability, no. 9,
pp. 2750,
July 1999. [Online]. Available:
http://www.cs.ukc.ac.uk/pubs/1999/747
B. Legeard, F. Peureux, and M. Utting, Automated
boundary testing from Z and B, in Proc. of the Int. Conf.
on Formal Methods Europe, FME02, ser. LNCS, vol.
2391. Copenhaguen, Denmark: Springer, July 2002,
pp. 2140.
B. K. Aichernig, Mutation Testing in the Refinement
Calculus, Formal Aspects of Computing, vol. 15, no.
2-3, pp. 280295, November 2003.
B. K. Aichernig and P. Pari-Salas, Test Case Generation by OCL Mutation and Constraint Solving, in Proceedings of the Fifth International Conference on Quality Software, Melbourne, Austrlia, September 2005.
S. Bracher and P. Krishnan, Enabling Security Testing from Specification to Code, in Fifth International
Conference on Integrated Formal Methods (IFM), ser.
LNCS, vol. 3771. Springer Verlag, November 2005,
pp. 150166.