Professional Documents
Culture Documents
2 3 3
1 2 2
the tool’s high-level description language.1 how this relates to the average case. For the latter, we show
We opt to build a small, but detailed, model of the sys- how obtaining best and worst-case adversaries (schedulers)
tem. The model comprises several components: one for for of the MDP can help to identify the scenarios in which the
each node in the network and one representing a scheduler best and worst cases occur. The next two sections describe
who chooses (nondeterministically) the order in which nodes these two illustrative properties in some detail; following
execute the protocol. The state of each node in the network this, we list some other properties that could be analysed.
includes its current view, information about which part of
the protocol it is currently executing and a buffer to store in- 4.1 Best, worst and average case behaviour
coming data from other nodes. The scheduler simply keeps We first study how the longest path length between nodes in
track of which nodes have sent data in the current round. the network varies over the execution of the protocol. This
For simplicity, we assume that the sending and updating of is done using PRISM properties of the form:
views is an atomic step, i.e. throughout the process of one
node sending its view to another, no other communication R{“path len”}min=? [ I=T ]
occurs in the network. R{“path len”}max=? [ I=T ]
Because of the detailed nature of the model and the corre- which represent the minimum/maximum expected value of
sponding state space size, we consider only very small sizes “path len” at time instant T . This assumes that we have
of network (N =3, 4) and fix a local view size of c = 2. It added a reward structure called “path len” to the PRISM
is possible, though, that anomalies observed in these small model, which associates with each state of the MDP a value
models will also be exhibited by networks of a more realistic representing the longest path length between any two nodes
size. As regards the initial configuration of the model (i.e. at that point (for the case where the graph is not connected,
the initial local views of each node), we assume that one we let “path len” be N ).
node is “public” and that all other nodes know the address These properties give the minimum and maximum values
of this node (but it is not aware of the others). This con- over all possible resolutions of nondeterminism in the MDP
figuration is suitably realistic and ensures that a connected which, in this model, means quantifying over all possible
network is possible. schedulings of the nodes. This allows us to determine the
For N =3, the model has 829 states and 946 transitions; best- and worst-case behaviour of the system in any even-
for N =4, it has 74,034 states and 87,410 transitions. These tuality. Since the gossip protocol makes random choices,
MDPs are constructed by PRISM in 1.73 seconds and 95.0 its execution under a particular scheduling is probabilistic.
seconds, respectively (on a 2GHz PC with 2GB RAM). Hence our use of minimum/maximum expected values.
It is also possible, with a simple modification of the PRISM
4. ANALYSIS AND RESULTS model, to compute the average value of the longest path
We used PRISM to analyse the performance of the gossip length over time. This is done by replacing nondetermin-
protocol described in the previous section, illustrating the ism in the scheduler component of the PRISM model with
kind of analysis that can be performed with probabilistic uniform probabilistic choices, yielding a DTMC instead of
model checking. We concentrate on how the topology of the an MDP. Although this is no longer an accurate model of
network induced by the local views of the nodes varies over the scheduling, it is interesting because the results computed
time, investigating first the maximum path length (longest from the DTMC model can be seen as the values that would
route between nodes) and then the time for the network to be obtained through simulation by averaging the results ob-
become connected. For the former, we demonstrate an anal- tained over a large number of simulation runs.
ysis of the best- and worst-case behaviour of the model, and Furthermore, we can also calculate the standard devia-
tion of the random variable corresponding to the longest
1 path length; again this is information that could be obtained
The full PRISM model can be found at:
http://www.prismmodelchecker.org/casestudies/gossip.php (approximately) through simulation. Computing this value
simistic view: in fact, the best and worst possible behaviour
4.5 is within the bounds given by the standard deviation. For
5 runs N =4, on the other hand, the worst-case (maximum values)
50 runs
are significantly higher. Also, for N =3, the average case
expected maximum path length at time T
500 runs
4 falls roughly half-way between the minimum and maximum
5000 runs
values, where as, for N =4, it is much closer to the best-case
(minimum) behaviour.
3.5 Lastly, to illustrate the relationship between the above
results and those obtained from discrete-event simulation,
in Figure 2 we have included the average results over 5, 50,
3 500 and 5,000 simulation runs for the network of 4 nodes.
The plots demonstrate that, as we increase the number of
simulation runs, the average values converge to the (average)
2.5
results for the DTMC model given in Figure 1(b).
2
4.2 Best and worst-case scheduling
One weakness with the property analysed in the previous
section is the notion of time used. Each time-step (as mea-
1.5 sured on the X-axis in the plots) corresponds to a single
0 200 400 600 800 1000 transition in the model. Because the model comprises a set
time (T)
of processes running in parallel this does not give an ac-
Figure 2: Simulation results (N =4). curate measure of elapsed time. Since the gossip protocol
proceeds in rounds of fixed time interval, however, we can
(exactly) with PRISM is done by adding a second reward improve this by considering the number of rounds.
structure, which associates each state of the model with the More precisely, we compute the (minimum and maximum)
square of the “path len” value, and then using the equiva- expected number of complete gossiping rounds required be-
lence σ(X)2 = E(X 2 ) − E(X)2 . fore the combined views of the nodes generate a connected
Figures 1(a) and 1(b) show the full set of these results for network. This is a desirable configuration for the network to
N =3 and N =4 nodes, respectively. The thicker solid lines reach since, when the local views do not form a connected
show the minimum and maximum expected longest path topology, the nodes have insufficient information to ensure
length after T time-steps, for a range of values of T . In that a message gets propagated to all other nodes in the
between these, the thinner solid line shows the average (i.e. network. The PRISM properties are:
expected) value for the same time points. The dashed lines
R{“num rounds”}min=? [ F “connected ” ]
indicate the standard deviation.
R{“num rounds”}max=? [ F “connected ” ]
The results demonstrate that there is a significant differ-
ence between the minimum and maximum values, i.e. be- where “num rounds” is a reward structure that assigns a
tween the best- and worst- case behaviour of the protocol. reward of 1 to transitions marking the end of a round and
Both values eventually stabilise at 1, for N =3, and 2, for “connected ” labels states in which a path exists between any
N =4 (in each case, this is the shortest possible longest path pair of nodes in the network.
length since the local views are of size two). As well as computing these measures, we use PRISM to
Note that, despite the discontinuities seen in the graphs, generate actual adversaries that result in the minimum and
these results are exact and have been computed for every maximum values.2 Since the only nondeterminism in the
time step. In fact, plots of this kind are typical for systems model is due to scheduling of the nodes (i.e. the order in
which operate in rounds, each one requiring multiple discrete which they forward their views to their neighbours), we can
time-steps. In the case of the maximum values for N =4 (Fig- extract from an adversary the corresponding scheduling.
ure 1(b)) we see that, although the longest path length is We consider first the network of three nodes. The min-
converging towards two, there are many small jumps where imum and maximum number of complete rounds required
it increases and then decreases again. This phenomenon for the local views to generate a connected network is 0
can be observed more clearly in Figure 1(c), which shows and 1 respectively. For comparison, in the DTMC model,
the same plots for a smaller range of time values. This be- where nondeterminism has been replaced by uniform ran-
haviour can be attributed to the fact that, within each round dom choice, the expected number of rounds is 0.667.
of the protocol, the adversary can schedule nodes in a mali- Figure 3 illustrates the sequences of node schedulings that
cious fashion such that the longest path length temporarily result in these minimum and maximum values. Initially, all
increases. Because of the design of the protocol, though, nodes can see node n2 , but no others. The best-case be-
the expected longest path length decreases as the rounds haviour (minimum expected number of complete rounds)
progress. Figure 1(c) also demonstrates that, although they can be obtained by first scheduling node n1 and then node
are not as pronounced, the same fluctuations occur for the n3 , after which n2 has added both n1 and n3 to its view and
other plots (average and minimum values). the network is connected. Since n2 has yet to be scheduled,
It is also interesting to observe the relationship between the minimum expected number of complete rounds is 0. For
the minimum and maximum values (obtained from the MDP) worst-case behaviour (maximum value), we can schedule n2
and the average and standard deviation values (obtained
from the DTMC). For the case where N =3, we see that the 2
We used a prototype extension of the tool that includes
average values and standard deviation give a slightly pes- this functionality.
n1 n2 n3 n4 n1 n2 n3 n4
schedule n1 schedule n1
n1 n2 n3 n4 n1 n2 n3 n4
schedule n3 schedule n3
n1 n2 n3 n4 n1 n2 n3 n4
schedule n2 schedule n2
choose n1 choose n3 schedule n4
n1 n2 n3 n4 n1 n2 n3 n4 n1 n2 n3 n4
schedule n2 schedule n2
schedule n4 schedule n4 choose n3 choose n4
n1 n2 n3 n4 n1 n2 n3 n4 n1 n2 n3 n4 n1 n2 n3 n4
n1 n2 n3 nk
n1 n2 n3
n1 n2 n3 schedule n1
Figure 5: Chain of k nodes.
schedule n1 n1 n2 n3
cases notice that, when node n4 is scheduled the view of n2
n1 n2 n3 schedule n2
is updated, causing the removal of n1 (the oldest descrip-
tor). Not also that, when node n2 is scheduled it makes a
schedule n3 n1 n2 n3 (random) selection between communicating with n1 or n3
since both are in its view at the time.
n1 n2 n3 schedule n3 For the minimum case (Figure 4(a)) we see that, if n2
chooses to gossip with n3 (i.e. the right-hand branch), then
n1 n2 n3 the network is complete by the end of the first round. If it
chooses n1 (i.e. the left branch) this is not the case (there
(a) minimum are no paths to n1 ) and further rounds of the protocol are
required. This leads to a (minimum) expected number of
(b) maximum rounds of 1.5. For the maximum case (Figure 4(b)), the
network is not connected under either choice and several
Figure 3: Scheduling for the 3 node network. further rounds are required.
The properties analysed in both this and the previous
before either n1 or n3 is scheduled. This adds no new infor- section demonstrate a considerable discrepancy between the
mation to the local views and means that a complete gossip- minimum and maximum values. To give a simple intuitive
ing round is required before the network becomes connected. explanation for this, consider a chain of nodes of length k
For the network consisting of four nodes, we find that (illustrated in Figure 5) in which n1 is trying to pass a mes-
the minimum and maximum expected number of complete sage to node nk . Suppose that all nodes are scheduled in
rounds before connectivity are 1.5 and 4.5 respectively. For each round and that nodes send only messages to their right-
comparison, the expected number of rounds for the DTMC hand neighbour (ni only sends messages to ni+1 ). Then, the
model is 2.788 which, unlike in the case of the three node scheduling n1 , n2 , . . . , nk would propagate the message to
network, is closer to the minimum than the maximum. node nk in a single round but the scheduling nk , nk−1 , . . . ,
This case is more complex than the three node network. n1 would require k−1 rounds to achieve this.
Since more than one round may be required before the net- As the number of nodes in the network increases, so does
work becomes connected and the number of possible neigh- the amount of nondeterminism present in the model. Hence,
bours exceeds the size of the view, descriptors can be dropped the potential influence of the scheduling (the difference be-
from the views as more recent information becomes avail- tween the minimum and maximum values) is also likely to
able. Furthermore, we must consider a node’s choice of who increase. As the comparison of the cases of three and four
to send data to. Figure 4 shows part of the scheduling (the nodes suggests, however, it may also be the case that, for
first gossiping round) that can result in the minimum and larger numbers of nodes, the average behaviour is closer to
maximum expected number of complete rounds. In both the best-case behaviour than the worst-case.
4.3 Other properties [3] R. Bakhshi, F. Bonnet, W. Fokkink, and
The PRISM model we have constructed could also be used B. Haverkort. Formal analysis techniques for gossiping
to analyse a variety of other properties. For example: protocols. ACM SIGOPS Operating Systems Review,
41(5):28–36, 2007.
• the maximum probability that a connected network [4] A. Bianco and L. de Alfaro. Model checking of
eventually becomes disconnected; probabilistic and nondeterministic systems. In Proc.
• the minimum probability node ni can communicate FST&TCS’95, volume 1026 of LNCS, pages 499–513.
with node nj after k gossiping rounds or t time steps; Springer, 1995.
[5] M. Duflot, M. Kwiatkowska, G. Norman, and
• the probability that node ni can communicate with D. Parker. A formal analysis of Bluetooth device
node nj before it can communicate with node nk ; discovery. Int. Journal on Software Tools for
Technology Transfer, 8(6):621–632, 2006.
• the maximum expected number of updates to the par-
tial views before the network is connected. [6] A. Fehnker and P. Gao. Formal verification and
simulation for performance analysis for probabilistic
Furthermore, the model could easily be adapted to study the broadcast protocols. In Proc. ADHOC-NOW’06,
effect on performance of a variety of other factors such as volume 4104 of LNCS, pages 128–141. Springer, 2006.
failures of network links and the dynamic addition/removal [7] H. Hansson and B. Jonsson. A logic for reasoning
of additional nodes. It could also be modified to study about time and reliability. Formal Aspects of
possible ways of preventing the potential inefficiencies high- Computing, 6(5):512–535, 1994.
lighted by our analysis. We could, for example, investigate [8] A. Hinton, M. Kwiatkowska, G. Norman, and
the performance of a modified version of the gossip protocol D. Parker. PRISM: A tool for automatic verification
in which the delays between each node’s execution of the of probabilistic systems. In Proc. TACAS’06, volume
protocol is also randomised. 3920 of LNCS, pages 441–444. Springer, 2006.
[9] M. Jelasity, S. Voulgaris, R. Guerraoui, A. Kermarrec,
5. CONCLUSIONS and M. van Steen. Gossip-based peer sampling. ACM
Trans. Computer Systems, 25(3), 2007.
In this paper, we have shown that probabilistic model check-
ing can be used to find interesting properties of gossiping [10] J. Kemeny, J. Snell, and A. Knapp. Denumerable
protocols that would be difficult to discover using alterna- Markov Chains. Springer-Verlag, 2nd edition, 1976.
tive analysis techniques such as simulation. Although using [11] M. Kwiatkowska, G. Norman, and D. Parker.
probabilistic model checking limits the size of the networks Game-based abstraction for Markov decision
that can be analysed, these ‘small’ networks can still high- processes. In Proc. QEST’06, pages 157–166. IEEE
light interesting behaviour that may also occur in more re- Press, 2006.
alistic network configurations. The results we have obtained [12] M. Kwiatkowska, G. Norman, and D. Parker.
demonstrate that modelling unknown choices (the schedul- Symmetry reduction for probabilistic model checking.
ing of the nodes in each gossiping round) with randomness In Proc. CAV’06, volume 4114 of LNCS, pages
causes a loss of information: the average case can be very 234–248. Springer, 2006.
different from the extreme (best/worst) cases. [13] M. Kwiatkowska, G. Norman, D. Parker, and
Possible future work includes extending the approach to J. Sproston. Performance analysis of probabilistic
include timing characteristics by using probabilistic timed timed automata using digital clocks. Formal Methods
automata [13, 15]. These can be considered as an extension in System Design, 29:33–78, 2006.
of MDPs that allows the modelling of real-time character- [14] M. Kwiatkowska, G. Norman, and J. Sproston.
istics, in addition to probabilistic and nondeterministic be- Probabilistic model checking of the IEEE 802.11
haviour. Another direction would be to employ techniques wireless local area net work protocol. In Proc.
such as abstraction [11, 18] and symmetry reduction [12] to PAPM/PROBMIV’02, volume 2399 of LNCS, pages
enable the analysis of larger network configurations. 169–187. Springer, 2002.
[15] M. Kwiatkowska, G. Norman, J. Sproston, and
Acknowledgments F. Wang. Symbolic model checking for probabilistic
The authors are supported in part by the EPSRC grants timed automata. Information and Computation,
EP/D07956X and EP/D076625. This work was initiated at 205(7):1027–1077, 2007.
the “Two Decades of Probabilistic Verification - Reflections [16] PRISM web site. www.prismmodelchecker.org.
and Perspectives” meeting at the Lorentz Center, Leiden, [17] J. Rutten, M. Kwiatkowska, G. Norman, and
organised by the NWO/DFG-funded VOSS II project. D. Parker. Mathematical Techniques for Analyzing
Concurrent and Probabilistic Systems, P. Panangaden
and F. van Breugel (eds.), volume 23 of CRM
6. REFERENCES Monograph Series. AMS, 2004.
[1] C. Baier, B. Haverkort, H. Hermanns, and J.-P. [18] B. Wachter, L. Zhang, and H. Hermanns. Probabilistic
Katoen. Model-checking algorithms for model checking modulo theories. In Proc. QEST’07,
continuous-time Markov chains. IEEE Trans. Software pages 129–140. IEEE Press, 2007.
Engineering, 29(6):524–541, 2003. [19] F. Werner and P. Schmitt. Analysis of the
[2] C. Baier and M. Kwiatkowska. Model checking for a authenticated query flooding protocol by probabilistic
probabilistic branching time logic with fairness. means. In Proc. WONS’08, pages 101–104, 2008.
Distributed Computing, 11(3):125–155, 1998.