You are on page 1of 37

Vi rtual Memory

PETER J. DENNING
Princeton University,* Princeton, New Jersey
\
The need for automatic storage allocation arises from desires for program
modularity, machine independence, and resource sharing. Virtual memory is an
elegant way of achieving these objectives. In a virtual memory, the addresses a
program may use to identify information are distinguished from the addresses the
memory system uses to identify physical storage sites, and program-generated
addresses are translated automatically to the corresponding machine addresses.
Two principal methods for implementing virtual memory, segmentation and
paging, are compared and contrasted. Many contemporary implementations have
experienced one or more of these problems: poor utilization of storage, thrashing,
and high costs associated with loading information into memory. These and
subsidiary problems are studied from a theoretic view, and are shown to be
controllable by a proper combination of hardware and memory management
policies.
Key words and phrases: virtual memory, one-level store, memory allocation,
storage allocation, dynamic storage allocation, segmentation, paging, replacement
algorithms, storage fragmentation, thrashing, working set
CR categories: 4.3, 6.2
INTRODUCTION
Fr o m t he earl i est days of el ect roni c com-
put i ng i t has been recogni zed t hat , because
fast -access st or age is so expensi ve, c omput e r
memor i es of ve r y l arge over al l capaci t y mus t
be or gani zed hi er ar chi cal l y, compr i si ng at
l east t wo levels, " ma i n me mo r y " and "auxi l -
i ar y me mo r y . " A pr ogr a m' s i nf or mat i on
(i.e. i nst r uct i on code and da t a ) can be
r ef er enced onl y when i t resi des in ma i n
me mor y; t hus, i nf or mat i on havi ng i mmedi -
at e l i kel i hood of bei ng r ef er enced mus t
resi de in mai n me mor y, and all ot her i nfor-
ma t i on in auxi l i ar y me mor y. The storage
allocation problem is t ha t of det er mi ni ng, at
each mome nt of t i me, how i nf or mat i on shal l
be di s t r i but ed a mong t he levels of me mor y.
Dur i ng t he ear l y year s of comput i ng,
each pr ogr a mme r had t o i ncor por at e st or age
* Department of Electrical Engineering. This work
was supported in part by National Aeronautics
and Space Administration Grant NGR-31-001-170.
al l ocat i on pr ocedur es i nt o his pr ogr a m
whenever t he t ot a l i t y of i t s i nf or mat i on was
expect ed t o exceed t he size of ma i n me mor y.
These pr ocedur es were r el at i vel y s t r ai ght -
f or war d, a mount i ng t o di vi di ng t he pr ogr a m
i nt o a sequence of " s e gme nt s " whi ch woul d
" o v e r l a y " (i.e. repl ace) one a not he r in mai n
me mor y. Si nce t he p r o g r a mme r was i nt i -
ma t e l y f ami l i ar wi t h t he det ai l s of bot h t he
machi ne and his al gor i t hm, i t was possi bl e
for hi m t o devi se efficient " ove r l a y se-
quences " wi t h r el at i ve ease.
The pi ct ur e began t o c h a n g e ma r k e d l y
af t er t he i nt r oduct i on of hi gher l evel pr o-
gr a mmi ng l anguages in t he mi d-1950s. Pr o-
gr a mme r s wer e encour aged t o be mor e
concer ned wi t h pr obl em- s ol vi ng and less
concer ned wi t h machi ne det ai l s. As t he
compl exi t y of t hei r pr ogr a ms grew, so gr ew
t he magni t ude of t he st or age over l ay pr ob-
l em. I ndeed, b y t he l at e 1950s i t was cl ear
t ha t pr ogr a m oper at i ng efficiency coul d
suffer gr eat l y under poor over l ay st r at egi es,
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
154 * Peter J. Denning
CONTENTS
Introduction 153-157
Basic System Hardware 157
Definition of Virtual Memory 157-159
Manual Versus Automatic Memory Management 159-
160
Implementation of Virtual Memory 16O-165
Segmentation
Paging
Segmentation and Paging
Storage Utilization 165-172
Placement Policies
Overflow and Compaction
Fragmentation
Page Size
Compression Factor
Comparison of Paged and Nonpaged Memories 172
Demand Paging 173-177
Paging Drum
Cost
Program Behavior and Memory Management 177-183
Replacement Algorithms
Optimal Paging Algorithms
The Principle of Locality and the Working Set Model
Multiprogramming and Thrashing
Program Structure 183-184
Hardware Support 184-186
Conclusions 186-187
References 187-189
and it was generally agreed t hat storage
allocation had become a problem of central
importance. But , since programmers were
shielded from machine details by .program-
ming languages, it was increasingly diffi-
cult t o persuade t hem t o expend t he now
relatively large effort required to devise good
overl ay sequences. This situation led to t he
appeal of computers havillg very large main
memories [M5].
Two divergent schools of t hought about
solutions emerged. These have come to be
known as t he static (preplanned) and dy-
namic approaches t o storage allocation.
These two approaches differ on their assump-
tions about t he most fundament al aspect of
the storage allocation problem, prediction,
bot h (1) of t he availability of memory
resources, and (2) of certain properties of a
program' s "reference string, " i.e. its se-
quence of references t o information.
The static approach assumes that. (1) is
either given or can be prespecified, and t hat
(2) can be determined either by preprocess-
ing t he program and recording its reference
string, or by examining the st ruct ure of its
text during compilation [C5, K1, O1, R1, R4].
The dynami c approach assumes t hat (1)
cannot (or ought not) be prespecified, and
t hat (2) is determinable only by observing
the program during execution; t he memory
space in use by a program should grow and
shrink in accordance with the program' s
needs [$1]. Comput er and programming
systems during t he 1960s have so evolved
t hat , in a great many cases, neither memory
availability nor program behavior are suffi-
ciently predictable t hat t he static approach
can provide a reasonable solution. The
reasons for this can be classed as program-
ming reasons and system reasons.
To understand t he programming reasons,
it is useful t o distinguish t wo concepts:
address space, t he set of identifiers t hat may
be used by a program to reference informa-
tion, and memory space, the set of physical
main memory locations in which information
items may be stored. In early comput er sys-
tems t he address and memory spaces were
t aken t o be identical, but in many con-
t emporary systems these spaces are dis-
Computing Surveys, Vol. 2, No. 3, September 1970
tinguished. This distinction has been made
to facilitate t he event ual achievement of
three objectives.
1. Machine independence. There is no a
priori correspondence between address
space and memory space.
The philosophy behind machine inde-
pendence is: I t relieves t he programmer
of the burden of resource management,
allowing him to devote his efforts fully t o the
solution of his problem; it permits equip-
ment changes in t he comput er syst em with-
out forcing reprogramming; and it permits
the same program to be run at different
installations.
2. Program modularity. Programs may
be constructed as collections of sepa-
rat el y compilable modules which are
not linked together to form a complete
program until execution time.
The philosophy behind program modul ari t y
is: I t enables independent compilation, test-
ing, and document at i on of the components
of a program; it makes it easier for several
programmers to work i ndependent l y on
parts of the same job; and it enables t he
modules constructed for one j ob t o be used
in another, i.e. building on the work of
others [D4, D5, D10, Dl l , D12, D13,
P2, R3, W4].
3. List processing. Languages (e.g. LisP)
having capability for handling problems
involving st ruct ured dat a are in-
creasingly i mport ant .
As we suggested earlier, these three pro-
gramming objectives invalidate reliable pre-
dictability, upon which static st orage allo-
cation is predicated. The mechanisms t hat
implement machine independence cannot
(by definition) establish a correspondence
between addresses and locations until exe-
cution time, much too late for a programmer
or a compiler t o preplan memory use. Pro-
gram modul ari t y makes it impossible for t he
compiler of a module to know either what
modules will constitute t he remainder of a
program or (even if it could know) what
their resource requirements might be. List
processing languages employ dat a struc-
tures whose sizes var y during execution and
Virtual Memory 155
which, by their ver y nature, demand dy-
namic storage allocation.
The major syst em reasons compelling
dynamic storage allocation result from cer-
tain objectives arising principally in multi-
programming and time-sharing systems: (1)
the ability to load a program into a space of
arbi t rary size; (2) t he abi l i t y t o run a
partially loaded program; (3) t he ability to
var y t he amount of space in use by a given
program; (4) the ability t o "rel ocat e" a
program, i.e. t o place i t in any available
part of memory or to move i t around during
execution; (5) t he ability to begin running a
program within certain deadlines; and (6)
t he ability t o change syst em equi pment
wi t hout having to reprogram or recompile.
Program texts prepared under the static
approach require t hat t he (rather inflexible)
assumptions about memory availability, On
which t hey are predicated, be satisfied be-
fore t hey can be run. Such texts are generally
incompatible with these six objectives.
Even within t he dynamic storage alloca-
tion camp there was disagreement. One
group held t hat the-programmer, being best
informed about his own algorithm' s opera-
tion, should be in complete control of storage
allocation. He would exercise this control
by calling on syst em routines which would
"al l ocat e" and "deal l ocat e" memory regions
on his behalf. This thinking is at least
partially responsible for the block struc-
t ure and st ack implementation of t he
ALGOL programming language (1958) and
subsequently t he ALgoL-oriented Burroughs
computers. I t has also influenced t he imple-
mentation of list-processing languages [B8,
C4, K4].
The other group in the dynamic storage
allocation camp advocat ed a very different
approach: automatic storage allocation. Their
thinking was influenced by their belief
t hat complicated programs beget storage
allocation problems so complicated t hat
most programmers could not afford t he time
t o manage memory well, and most particu-
larly by their belief t hat multiprogram-
ming would soon be a concept of great
importance. Because t he availability in
main memory of particular part s of address
Computing Surveys, Vol. 2, No. 3, September 1970
156 Peter J. Denning
space may be unpredictable under multi-
programming, a programmer' s ability to
allocate and deallocate storage regions may
be seriously impaired. Realizing t hat the
principal source of difficulty was t he small
size of programmable main memory, this
group advanced the concept of a o~e-level
store. I n 1961 a group at MI T [M5] pro-
posed the construction of a computer having
several million words of main memory (an
amount t hen considered vast) so t hat the
storage allocation problem would vanish.
Economic reasons prevented this from ac-
t ual l y being realized.
I n 1961 the group at Manchester, Eng-
land, published a proposal for a one-level
store on the Atlas computer [F3, K3], a
proposal t hat has had profound influence
on computer System architecture. Their
idea, known now as virtual memory, gives
t he programmer the illusion t hat he has a
very l ar ge main memory at his disposal,
even though t he computer act ual l y has a
relatively small main memory. At the heart
of their idea is the notion t hat "address"
is a concept distinct from "physical loca-
t i on. " I t becomes the responsibility of the
computer hardware and software auto-
matically and propitiously to move infor-
mat i on into main memory when and only
when it is required for processing, and to
arrange t hat program-generated addresses
be directed to the memory locations t hat
happen to contain the information addressed.
The problem of storage allocation (for ob-
jects represented in virtual memory) thus
vanishes completely from the programmer' s
purview and appears in t hat of t he com-
puter system. By basing memory use on
system-observed actual use of space, rat her
t han (poor) programmer estimates of space,
virtual memory is potentially more efficient
t hat preplanned memory allocation, for it
is a form of adaptive system.
By the mid-1960s the ideas of virtual
memory had gained widespread acceptance,
and had been applied to the internal design
of many large processors--IBM 360/85 and
195, CDC 7600, Burroughs B6500 and later
series, and GE 645, to name a few. The
fact of its acceptance testifies to its general-
i t y and elegance.
The foregoing discussion has summarized
the ideas leading to the virtual memory
concept. By distinguishing between ad-
dresses and locations, and aut omat i ng stor-
age allocation, virtual memory facilitates
certain programming and system design
objectives especially i mport ant in multipro-
gramming and time-sharing computers. The
discussion in the remainder of this paper
divides into two general areas: the mecha-
riisms for effecting virtual memory, and the
policies for using the mechanisms. The prin-
cipal mechanisms are: segmentation, under
which the address space is organized into
variable size "segment s" of contiguous
addresses; and paging, under which the
address space is organized into fixed size
"pages" of contiguous addresses. We shall
compare and contrast these two mecha-
nisms and show why systems using some
form of paging are predominant.
Although it has some very i mport ant
advantages, virtual memory has not been
without its problems. There are four of
particular interest. (1) Many programmers,
in their illusion t hat memory is unlimited,
are undul y addicted to the old idea t hat time
and space may be traded, in the sense
t hat a program' s running time may be re-
duced if there is more programmable
memory space available. But space in a
vi rt ual memory may be an illusion; un-
necessarily large and carelessly organized
programs may generate excessive overhead
in the aut omat i c storage allocation mecha-
nism, inevitably detracting from tile effi-
ciency of program operation. Nonetheless,
as programmers and language designers
gain experience with virtual memory, this
problem should disappear. (2) Many paged
systems suffer severe loss of usable st orage--
"fragment at i on"--because storage requests
must be rounded up to the nearest integral
number of pages. (3) Many time-sharing
systems using "pure demand paging" (a
policy under which a page is loaded into
main memory only after an at t empt ed
reference to it finds it missing) experience
severe costs as a program' s working pages
are loaded singly on demand at the st art
of each time quant um of execution. (4)
Many systems have shown extreme sensi-
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
t i vi t y t o " t hr as hi ng, " a phenomenon of
compl et e per f or mance collapse t ha t ma y
occur under mul t i pr ogr ammi ng when mem-
or y is over commi t t ed. We shall demons t r at e
t ha t t hese pr obl ems ma y be cont r ol l ed if
vi r t ual me mor y mechani sms are gover ned
by sound st rat egi es.
The r eader shoul d not e t ha t t hese f our
obser ved i nadequaci es of ma ny cont em-
por ar y syst ems r esul t not f r om i l l -concei ved
mechani sms, but f r om i l l -concei ved policies.
These difficulties have been so publ i ci zed
t ha t an unsuspect i ng newcomer ma y be
led er r oneousl y t o t he concl usi on t ha t vir-
t ual memor y is folly. Qui t e t he cont r ar y;
vi r t ual memor y is dest i ned t o occupy a
pl ace of i mpor t ance in comput i ng f or ma ny
year s t o come.
BASIC SYSTEM HARDWARE
As our basic comput er syst em, we t ake t ha t
shown in Fi gur e 1. The memor y syst em
consists of t wo levels, mai n memor y and
auxi l i ary memor y. One or mor e processors
have di rect access t o mai n memor y, but not
t o auxi l i ary memor y; t her ef or e i nf or mat i on
ma y be processed onl y when in mai n mem-
ory, and i nf or mat i on not bei ng processed
ma y reside i n auxi l i ary memor y. Fr om now
on, t he t e r m " me mo r y " specifically means
"mai n me mor y. "
Ther e are t wo t i me par amet er s of i nt er est
here. The first, known as " me mor y reference
t i me, " is measur ed bet ween t he moment s
at whi ch references t o i t ems in memor y are
i ni t i at ed by a processor; i t is composed of
del ays resul t i ng f r om memor y cycl e t i me,
f r om i nst r uct i on execut i on t i me, f r om "in-
t er f er ence" by ot her processors at t empt i ng
t o reference t he same memor y modul e
si mul t aneousl y, and possi bl y also from
swi t chi ng processors among programs. We
t ake t he average memory reference time t o be
,5. The second t i me par amet er , known as
" t r a ns por t t i me, " is t he t i me r equi r ed t o
compl et e a t r ans act i on t ha t moves i nfor-
mat i on bet ween t he t wo levels of memor y;
i t consists of del ays resul t i ng f r om wai t i ng
in queues, f r om wai t i ng f or t he r equest ed
i nf or mat i on t r ansf er t o finish, and possi bl y
Virtual Memory 157
PROCESSORS
Fie,. 1. Basic system hardware
AUXILIARY
MEMORY
also f r om wai t i ng f or r ot at i ng or movabl e
devi ces t o be posi t i oned ( " l at ency t i me") .
We t ake t he average transport time t o be T.
Si nce mai n memor i es ar e or di nar i l y elec-
t r oni cal l y accessed and auxi l i ar y memor i es
mechani cal l y accessed, ,5 is t ypi cal l y 1
~sec and T is t ypi cal l y at l east 10 msec.
Thus speed r at i os (T/ h) in t he or der of 104
or mor e are not uncommon.
Mai n me mor y ma y be r egar ded as a
l i near a r r a y of "l ocat i ons , " each servi ng
as a st or age site f or an i nf or mat i on i t em.
Each l ocat i on is i dent i fi ed by a uni que
" me mor y addr ess. " I f t he me mor y cont ai ns
m l ocat i ons, t he addresses are t he i nt egers
0, 1, . - . , m - 1. I f a i s an address, t he
i t em st or ed in l ocat i on a is cal l ed t he "con-
t ent s of a, ' ; and is denot ed c(a). Under
pr ogr am cont rol , a processor gener at es a
sequence of "r ef er ences" t o me mor y loca-
t i ons, each consi st i ng of an addr ess and a
c omma nd t o " f e t c h" f r om or " s t o r e " i nt o
t he desi gnat ed l ocat i on.
DEFINITION OF VIRTUAL MEMORY
As ment i oned earlier, vi r t ual memor y ma y
be used t o gi ve t he pr ogr ammer t he illu-
sion t ha t memor y is much l arger t ha n i n
r eal i t y. To do t hi s, i t is necessar y t o allow
t he pr ogr ammer t ~ use a s e t - of addresses
di fferent f r om t ha t pr ovi ded by t he me mor y
and t o pr ovi de a mechani sm f or t r ansl at i ng
pr ogr am- gener at ed addresses i nt o t he cor-
r ect me mor y l ocat i on addresses. An address
used by t he pr ogr ammer is called a " n a me "
or a " vi r t ual addr ess, " and t he set of such
names is called t he address space, or name
space. An address used b~,: t he me mor y is
called a " l ocat i on" or " me mo r y addr ess, "
Computing Surveys, Vol. 2, No. 3, September 1970
158 Peter J. Denning
!
2
4
7
i I
m - !
M0f~Y SPACE
ADDRESS SPACE
Fi G. 2. Mappi ng f r om name t o m em or y space
and t he set of such l ocat i ons is cal l ed t he
memory space. F o r f ut ur e reference we
denot e t he address space by N -- {0, 1,
. . , n - 1} and t he me mor y space b y
M = { 0 , 1 , . - . , m - 1} and we assume
n > m unless we say ot herwi se.
Si nce t he address space is r egar ded as a
col l ect i on of potentially usabl e names for
i nf or mat i on i t ems, t her e is no r equi r ement
t ha t e ve r y vi r t ual address " r e pr e s e nt " or
" c ont a i n" any i nf or mat i on.
The price t o be pai d f or t her e bei ng no a
pr i or i cor r espondence bet ween vi r t ual ad-
dresses and me mor y l ocat i ons is increa,sed
compl exi t y in t he addressi ng mechani sm.
We mus t i ncor por at e a way of associ at i ng
names wi t h l ocat i ons dur i ng execut i on.
To t hi s end we define, f or each mome nt of
t i me, a f unct i on f : N- + M U {4~} such t ha t
' i f i t em a is in M at l ocat i on a' ,
f(a) = if i t em a is missing f r om M.
Thi s f unct i on f is known as t he address map,
or t he address-translation function.
For reasons gi ven earlier, i t is t o our
advant age t o make n much l arger t ha n m,
but t hi s is not necessary. Eve n i f n _~ m,
vi r t ual st or age coul d hel p wi t h t he relocation
problem [Dl l ], i.e. t ha t of movi ng i nforma-
t i on ar ound in memor y.
Fi gur e 2 gives an exampl e of a mappi ng
f , where a line (a, a') f or a in N and a'
i n M i ndi cat es t ha t i t em a is st or ed in loca-
t i on a' , and t he absence of a line i ndi cat es
t ha t i t em a is not pr esent in M. Fi gur e 3
shows how a har dwar e devi ce i mpl ement i ng
f coul d be i nt erposed bet ween t he processor
and t he memor y t o handl e t he mappi ng
aut omat i cal l y. Not e t hat , by vi r t ue of t he
mappi ng f , t he pr ogr ammer ma y be gi ven
t he illusion t ha t i t ems consecut i ve in N are
st or ed consecut i vel y i n M, even t hough t he
i t ems ma y i n f act be st or ed in ar bi t r ar y
order. Thi s pr ope r t y of address maps is
known as "art i fi ci al cont i gui t y" [R3].
The mappi ng devi ce, when pr esent ed wi t h
name a, will gener at e a p = f(a) if i t em a is
pr esent i n M, and a missing-item faul t
ot herwi se. The f aul t will i nt er r upt t he
processi ng of t he pr ogr am unt i l t he missing
i t em can be secured f r om auxi l i ary me mor y
and pl aced in M at some l ocat i on a' (which
t akes one t r ans por t t i me) ; t he address map
f is t hen redefi ned so t ha t f(a) = a' , and
t he reference ma y be compl et ed. I f M is full,
some i t em will have t o be r emoved t o make
way f or t he i t em ent er i ng, t he par t i cul ar
i t em bei ng chosen at t he di scret i on of t he
replacement rule (if i t em b is ent er i ng and t he
r epl acement rul e chooses t he repl ace i t em a,
where a p -- f(a), t hen t he address map is
redefi ned so t ha t f(b) becomes a' and
f(a) becomes ). Cont r as t ed wi t h t he re-
pl acement rul e, whi ch deci des whi ch i t ems
t o r emove, are t he fetch rule, which deci des
when an i t em is t o be l oaded, and t he
placement rule, whi c h decides where t o
place an i t em. I f no act i on is t aken t o l oad
an i t em i nt o M unt i l a f aul t for i t occurs,
t he f et ch rul e is known as a demand rule;
ot herwi se, if act i on is t aken t o l oad an i t em
before i t is referenced, t he f et ch rule is
known as a nondemand or anticipatory rule.
ADDRESS TRANSLATION MECHANISM
-r 1
I MAP TABLE f I
'
O'
P t AOORESS } MEMORY
I
OPERAT I ON :
a loaded into VA
if ath entry of j blank, missing-item fault
a' loaded into MA
FIG. 3. Implementation of address map
CompuLi ng Surveys, Vol. 2, No. 3, Sept ember 1970
Consi der briefly t he i mpl ement at i on of
t he address map f. The simplest implemen-
t at i on t o visualize, called direct mappi ng,
is a t abl e cont ai ni ng n ent ri es; t he at h
ent r y cont ai ns a' whenever f (a) = a r, and
is bl ank (i.e. cont ai ns t he symbol ~) other-
wise. If, as would nor mal l y be t he case, n
is much great er t han m, t hi s t abl e woul d
cont ai n a great many (i.e. n - m) bl ank
entries. A much more efficient way t o repre-
sent f is t o create a t abl e cont ai ni ng onl y
t he mapped addresses; t he t abl e cont ai ns
exact l y t he pairs (a, a ~) for which f (a) = a'
and no pair (a, ~), and t hus cont ai ns at
most m entries. Such a t abl e is more com-
pl i cat ed to use; when present ed wi t h name
a, we must search unt i l we find (a, a' ) for
some a' , or unt i l we have exhaust ed t he
t abl e. Har dwar e associative memories are
nor mal l y empl oyed for storage of these
mappi ng tables, t her eby maki ng t he search
operat i on qui t e efficient. (An associative, or
"cont ent - addr essabl e, " memor y is a memor y
device which stores in each cell i nf or mat i on
of t he form (k, e),where k is a " k e y " and e
an " e nt r y. " The memor y is accessed by
present i ng i t wi t h a key k; if some cell con-
t ai ns (k, e) for some e, t he memor y ret urns
e, otherwise i t signals " not f ound. " The
search of all t he memor y cells is done si-
mul t aneousl y so t hat access is rapid. )
MANUAL VERSUS AUTOMATIC MEMORY
MANAGEMENT
The discussion in t he I nt r oduct i on reviewed
t he mot i vat i on for aut omat i c st orage allo-
cat i on from a qual i t at i ve view. Before open-
ing t he discussion of met hods for imple-
ment i ng and regul at i ng vi rt ual memor y, we
shoul d like t o mot i vat e aut omat i c st orage
allocation from a more quant i t at i ve view.
The quest i on before us is: How well does
aut omat i c storage allocation compet e wi t h
manual ?
Al t hough t he l i t erat ure cont ai ns sub-
st ant i al amount s of experi ment al i nforma-
t i on about program behavi or under auto-
mat i c st orage management [B3, B9, C3,
F2, F3, F4, K5, O2, $2], aut hors have reached
conflicting conclusions. Ma n y of t hese
Vi r t ual Me mor y 159
experi ment s addressed t he quest i on " How
do programs behave under given aut omat i c
st orage al l ocat i on policies?" but not t he
quest i on at hand, " How does aut omat i c
st orage allocation compare wi t h manual ?"
Exper i ment s for t he former quest i on are
clearly of a di fferent nat ur e t han t hose for t he
l at t er. Therefore, at t empt s t o make in-
ferences about t he l at t er from dat a gat her ed
about t he former are bound t o resul t i n
conflicting conclusions. The following dis-
cussion is based on a paper by Sayre [$2],
who has summari zed and i nt er pr et ed t he
work of Br awn and Gust avson [B9], for
these appear t o be t he onl y publ i shed works
addressing t he l at t er quest i on.
I f t he name space N is l arger t han t he
memor y space M, i t is necessary t o " f ol d"
N so t hat , when folded, N will " f i t " i nt o M.
Let g(b, t) denot e t he inverse of t he address
map f :
g(b, t) = I a if f (a) = b at t i me t,
undefi ned otherwise.
The address space N is said t o be fol ded if,
for some b and tl < t2, g(b, tl) # g(b, t2).
That is, t here is some memor y l ocat i on
wh i c h has been assigned t o more t ha n
one address duri ng t he course of a program' s
execution. Bet ween t he i nst ant s tl and t~,
a sequence of commands, mo v e o u t and
mo v e i n, must have been issued, which
caused g(b, tl) t o be repl aced by g(b, t2).
The name space N is manual l y fol ded if t he
programmer has prepl anned st orage alloca-
t i on, and has i nsert ed t he mo v e o u t and
mo v e i n commands i nt o t he program t ext
where needed. The name space is auto-
mat i cal l y fol ded if t he mo v e o u t and mo v e
i n commands are not in t he pr ogr am t ext ,
but i nst ead are generat ed by t he replace-
ment and fet ch rules, respectively, of t he
vi r t ual memor y mechani sm. Not e t ha t
manual l y fol ded t ext is i nt ended t o fit i nt o
some specific memor y space of size So,
whereas t he aut omat i cal l y folded t ext ma y
fit i nt o any nonempt y memor y space.
The quest i on before us now is: Can auto-
mat i c folding compet e wi t h manual folding?
I t is reasonabl y clear t ha t aut omat i c folding
shoul d be compet i t i ve when t he speed r at i o
Computing Surveys, Vol. 2, No. 3, September 1970
160 Peter J. Denning
T/ a between main and auxiliary memory is
small; but is it competitive when T/ h is
large (say, 104 or greater)? Sayre reports
affirmatively.
Brawn and Gusta~vson, Sayre tells us,
considered a number of programs represent-
ing a wide range of possible behaviors, and
the following experiment in a memory sys-
t em with T/& in excess o f 104. For a given
program, let Ta(so) denote t he t ot al running
time (execution and transport time) when
N is folded automatically into a memory
of size So, when a demand fetch rule and a
good replacement rule are in effect. Let
Tm(so) denote the t ot al running t i me when
N is folded manual l y for a memory of size
so. For t he programs considered,
0.8 < T~(so)/T~(so) < 1.7,
(i)
W[Ta(so)/Tm(so)] = 1.21,
where E[ ] denotes expected value. In
other words, automatic folding was (on t he
average) no more t han 21 percent less
efficient t han manual folding.
Now, let Ks(so) denote the number of
transports issued while the program ran
under the automatic folding conditions, and
Kin(so) denote the number of transports
under the manual folding conditions. For
t he programs considered,
0.6 < K~(so)/K,,(So) < 1.05,
E[K~(so)/Km(so)] = 0.94.
Thus the automatic folder (i.e. the virtual
memory) generally produced fewer moves
t han the manuM folder (i.e. the program-
mer). A similar result was observed by the
Atlas designers for a more restricted class of
programs [K3]. The advantage of manual
folding is t hat , unlike virtual memory with a
demand fetch rule, processing may be over-
lapped with transports This suggests t hat
anticipatory fetch rules might result in
ratios Ta(so)/T,~(so) consistently less t han
one [P1].
The experiments show also t hat t he auto-
matic folder is robust, i.e. it continues to
give good performance for memory sizes
well below the intended So. Specifically,
T~(s)/Tm(so) was found essentially constant
for a wide range of s, including s much less
t han So. In other words, a given program is
compatible with many memory sizes under
automatic folding, but only one under
manual.
As we shall see in the section on Program
Behavior and Memory Management, virtual
memory management mechanisms perform
most efficiently when programs exhibit good
locality, i.e. t hey t end to concentrate their
references in small regions of address space.
We shall define a measure of locality, the
working set of information, which will be
t he smallest set of virtual addresses t hat
must be assigned to memory locations so
t hat t he program may operate efficiently.
Sayre reports t hat the running time under
automatic folding, Ta(s0), can be very
sensitive to programmers' having paid at-
tention to endowing the programs with
small working sets, and relation (i) depends
on this having been done. Should program-
mers not pay at t ent i on to this, very large
Ta(so)/Tm(so) can occur. Sayre reports t hat
the costs of producing good manual l y :folded
text appear to exceed by 25 to 45 percent
the costs for producing nonfolded t ext with
good locality. Thus, one can tolerate as
much as 25 percent inefficiency in the auto-
matic folding mechanism before vi rt ual
memory begins to be less efficient t han
manual folding. Relations (i) indicates this
generally is the case.
On the basis of the experimental evidence,
therefore, we may conclude t hat the best
automatic folding mechanisms compete very
well (and may indeed outperform) the best
manual l y folded texts. Virtual memory is
t hus empirically justifiable.
IMPLEMENTATION OF VIRTUAL MEMORY
The table implementation for the address
mapping f described in the section on Defi-
nition of Virtual Memory is impractical,
because it would require a second memory
of size m t o store the mapping table. I n the
following sections we shall examine t hr e e
methods t hat result in a considerable re-
duction in the amount of mapping informa-
tion t hat must be stored. Each method
Computing Surveys, Vol, 2, No. 3, Sept ember 1970
groups information into blocks, a block
being a set of contiguous addresses in ad-
dress space. The entries in the mapping
table will refer now to blocks, which are far
less numerous t han individual addresses in
address space. The first met hod--segment a-
t i on- or gani zes address space into blocks
("segments") of arbitrary size. The second
method--paging--organizes memory space
into blocks ("pages") of fixed size. The third
met hod combines both segmentation and
paging.
Both segments and pages have names,
which can be used to loeate entries in the
map tables. Segment names are usually (but
not always) assigned by the programmer and
are interpreted by the software, and page
names are usually assigned by the system
and interpreted by the hardware. Segmenta-
tion and paging, when combined, form an
addressing system incorporating bot h levels
of names. Otherwise, the only essential
difference between the two schemes is
paging' s fixed block size.
Segmentation
Programmers normally require the ability
to group their information into content-
related or function-related blocks, and t he
ability to refer to these blocks by name.
Modern computer systems have four objec-
tives, each of which forces the system to
provide the programmer with means of
handling the named blocks of his address
space:
Program modularity. Each program
module constitutes a named block which is
subject to recompilation and change at any
time.
Varyi ng data structures. The size of
certain dat a structures (e.g. stacks) may
vary during use, and it may be necessary to
assign each such structure to its own, varia-
ble size block.
Protection. Program modules must be
protected against unauthorized access.
Sharing. Programmer A may wish to
borrow module S from programmer B, even
though S occupies addresses which A has
already reserved for other purposes.
These four objectives, together with
Vi rt ual Memory 161
machine independence and l i s t processing,
are not peculiar to virtual memory systems.
They were fought for in physical storage
during the late 1950s [W5]. ' Dynamic storage
allocation, linking and relocatable loaders
[M3], relocation and base registers [Dl l ],
and now virtual memory, ~ll result from t he
fight's having been won.
The segmented address space achieves these
objectives. Address space is regarded as a
collection of named segments, each being a
linear array of addresses. I n a segmented
address space, the programmer references
an information item by a two-component
address (s, w), in which s is a segment name
and w a word name within s. (For example,
t he address (3, 5) refers t o the 5th word in
in the 3rd segment.) We shall discuss shortly
how the address map must be constructed
to implement this.
By allocating each program module to its
own segment, a module' s name and internal
addresses are unaffected by changes in other
modules; thus the first two objectives may
be satisfied. By associating with each seg-
ment certain access privileges (e.g. read,
write, or instruction-fetch), protection may
be enforced. By enabling the same segment
to be known in different address spaces under
different names, the fourth objective may be
satisfied.
Figure 4 shows t he essentials of an ad-
dress translation mechanism t hat imple-
ments segmentation. The memory is a
linear array of locations, and each segment
is loaded in ent i ret y into a contiguous
region of memory. The address a at which
segment s begins is its base address, and the
number b of locations occupied by s is its
limit, or bound. Each ent ry in t he segment
table is called a descriptor; the sth descriptor
contains the base-limit information (a, b)
for segment s if s is present in memory, and
is blank otherwise. The steps performed in
forming a location address a' from a name
space address (s, w) are shown in Figure 4.
Note t hat a missing-segment fault occurs
if s it not present in memory, interrupting
program execution until s is placed in
memory; and an overflow fault occurs if w
falls outside the allowable limit of s. Pro-
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
162
SEGMENT
WORD
Peter J. Denning
SI~31WF~T TABLE ST
tl. ' "1
llA 7 LI MI T
MEMORY
AOORESS
OPERATION:
(s, w) loaded into segment and word registers
if sth entry of ST blank, missing-segment fault
if w > b, overflow fault
(a + w) loaded into MA
Fro. 4. Address translation for segmentation
t eet i on bi t s ( t he dar kened regi on i n t he
t abl e e nt r y of Fi gur e 4) can be checked
agai nst t he t ype of access bei ng a t t e mpt e d
(i.e. read, wri t e, or i nst r uct i on- f et ch) and a
pr ot ect i on f aul t gener at ed if a vi ol at i on is
det ect ed.
The segment t abl e can be st or ed in mai n
me mor y i nst ead of bei ng a component of
t he address t r ansl at i on mechani sm. Fi gur e 5
shows t he oper at i on of t he mappi ng mecha-
ni sm when t he segment t abl e is in memor y
st ar t i ng at l ocat i on A. The segment t abl e
is i t sel f a segment , known as t he descriptor
segment, and t he segment t abl e base regi st er
SEGMENT MEMORY
- - ~ AOORESS
SEGMENT
WORD BASE IdI~MOI~Y
REGISTER.
OPERATION :
(s, w) l oaded i nt o s egment a nd wor d regi st ers
(A + s) l oaded i nt o MA
c(A + s) fetched into MR
if MR blank, missing-segment fault
a : = base field of MR
b := limit field of MR
if w > b, overflow fault
(a + w) loaded into MA
Fro. 5. Segmentation with mapping table in mem-
ory
is known somet i mes as t he descriptor base
register.
I n t hi s case, each pr ogr am- gener at ed
access woul d i ncur t wo memor y- r ef er ences,
one t o t he segment t abl e, and t he ot her t o
t he segment bei ng r ef er enced; segment at i on
woul d t hus cause t he pr ogr am t o r un as slow
as hal f speed, a hi gh pri ce t o pay. A common
sol ut i on t o t hi s pr obl em i ncor por at es a
smal l hi gh speed associ at i ve me mor y i nt o
t he address t r ansl at i on har dwar e. Each
associ at i ve r egi st er cont ai ns an e nt r y (8, a, b)
and onl y t he most r ecent l y used such ent r i es
are r et ai ned t her e. I f t he associ at i ve memor y
cont ai ns (s, a, b) at t he moment (s, w) is t o
be referenced, t he i nf or mat i on (a, b) is
i mmedi at el y avai l abl e f or gener at i ng t he
l ocat i on address a' ; ot herwi se t he addi t i onal
reference t o t he segment t abl e is requi red.
I t has been f ound t ha t 8 t o 16 associ at i ve
regi st ers are sufficient t o cause pr ogr ams
t o r un at ve r y near l y full speed [$4]. ( The
exact number depends of course on whi ch
machi ne is under consi derat i on. )
Hi st or i cal l y, t he f our obj ect i ves discussed
at t he begi nni ng of t hi s sect i on haw~ been
pr ovi ded by "file s ys t ems , " whi ch per mi t
pr ogr ammer s t o mani pul at e named "f i l es"
and t o cont r ol decisions t ha t move t he m
bet ween mai n and auxi l i ary memor y. I n
principle, t her e is no need for t he pro-
gr ammer t o use a file syst em in a vi r t ual
me mor y comput er , since auxi l i ary memor y
is pr es umabl y hi dden f r om hi m and all his
i nf or mat i on ma y be per manent l y r epr esent ed
i n his address space. I n pr act i ce, most
cont empor ar y " vi r t ual memor y s ys t ems "
pr ovi de bot h a vi r t ual memor y and a file
syst em, t oget her wi t h "file processi ng primi-
t i ve s " t ha t oper at e out si de t he vi r t ual
memor y. I n t hese syst ems, a " s e gme nt " is a
"f i l e" t ha t has been moved f r om auxi l i ar y
memor y i nt o address space. Mul t i cs is t he
onl y document ed except i on t o t hi s [B7].
Among t he earl i est proposal s for segmen-
t at i on, t hough wi t hout t he use of an address
space, was Hol t ' s [H2]. Addressi ng schemes
ve r y similar t o t ha t gi ven in Fi gur e 4 were
first i mpl ement ed on t he Ri ce Uni ver si t y
Comput er [I1, I2] and on t he Bur r oughs
B5000 comput er [B10, M1]. Thi s i dea was
Computing Surveys, VoL 2, No. 3, September 1970
expanded, its implications explored, and a
strong case made in its favor by Dennis
[DIO-D12]. Details of implementing seg-
mentation and of combining segments into
programs during execution are given by
Arden et al. [AS], and again by Daley and
Dennis [D1]. Dennis and Van Horn [D13],
Johnston [J1], and also Wilkes [W4], place
segmentation in proper perspective among
all aspects of multiprocess computer systems.
Randell and Kuehner [R3] place segmenta-
tion in perspective among dynamic storage
allocation techniques, and provide details
for its implementation on various machines.
P a g i n g
Paging is another method for reducing
the amount of mapping information and
making virtual memory practical. Main
memory is organized into equal size blocks
of locations, known as page frames, which
serve as sites of residence for matching size
blocks of virtual addresses, known as pages.
The page serves as the unit both of informa-
tion storage and of transfer between main
and auxiliary memory. Each page frame will
be identified by its frame address, which is
the location address of the first word in the
page frame.
We suppose t hat each page consists of z
words contiguous in address space, and t hat
the address space N consists of n pages
{0, 1, 2, . . . , n - 1} (i.e. nz virtual ad-
dresses), and the memory space M consists
of m page frames {0, z, 2z, . . . , (m -- 1)z}
(i.e. mz locations). A virtual address a is
equivalent to a pair (p, w), in which p is a
page number and w a word number within
page p, according to the relation a = pz +
w, 0 <_ w < z, where p = [a/z], the
integer part of a/z, and w = R,(a), the
remainder obtained in dividing a by z. In
machines using binary arithmetic, the
computation t hat generates (p, w) from a is
trivial if z is a power of 2 [A5, Dl l ].
Figure 6 shows the essentials of the address
translation mechanism t hat implements
paging. The pt h ent ry of the page table con-
tains frame address p' if page p is loaded in
frame p' , and is blank otherwise. The steps
performed in forming location address a'
VtRTUAL
ADDRESS
Virtual Memory
P ~ T~. E PT
I
P
2 p' ~
1 63
MEMORY
ADDRESS
OP E RAT I ON:
a l o a d e d i n t o VA
p := [a/z]
w : = R , ( a )
if pth entry of PT blank, missing-page fault
(p' + w) loaded into MA
Fi~. 6. Address translation for paging
from virtual address a are shown in Figure 6.
Note t hat a missing-page fault occurs if p is
not present in memory, interrupting program
execution until p has been placed in an
available frame of memory. Protection bits
(the darkened area in the page table ent ry)
may be compared against the t ype of refer-
ence being attempted, and a protection
fault generated if a violation is detected.
As in the implementation of segmentation,
the page table can be stored in memory.
The modification of the address translation
mechanism follows the Lsame lines as Figure
5, and is not shown here. As before, program
operation may be speeded up by incorporat-
ing an associative memory into the address
translation mechanism t o retain t he most
recently used page table entries.
Paging was first used in t he Atlas compu-
ter IF3, K3], and is presently used by almost
every manufacturer in at least one of his
products [R3]. As with any virtual memory
system, it shields, the programmer from
storage allocation problems, and is therefore
susceptible to misuse; its performance has
generally been encouraging [A4, 02, P1, $2],
but occasionally discouraging [K6]. Because
paging has received a great deal of at t ent i on
in the literature, and its behavior nonetheless
tends not to be widely understood, we shall
Computing Surveys, ol. 2, No. 3, September 1920
k .> ,2 '~ : ~ : : ~ ' ~ , ~
164 Peter J. Denning
SEGMENT PAGE
TABLE ST TABL~E PT
I "
WORD
OPERATION :
MEMORY
ADORE:S$
(s, w) loaded into segment and word registers
if sth entry of ST blank, missing-segment fault
if w > b, overflow fault
p := [w/z]
w' := R~(w)
if pth entry of PTA blank, missing-page fault
(p " + w') loaded into MA
FIo. 7. Address translation for segmentation and
paging
devote most of the later sections of this paper
to it.
Segmentation and Paging
Because paging by itself does not alter the
linearity of address space, it does not achieve
the objectives t hat motivate segmentation.
Because segmentation by itself requires t hat
contiguous regions of various sizes be found
in me mor y t o store segments, it does not
result in the simple uniform t reat ment of
main memory afforded by paging. To under-
st and what is meant by "uniform t r eat ment "
of memory, compare t he problem of loading
new segment into memory with t hat of
loading a new page into memory. Loading a
segment requires finding an unallocated
region large enough to contain the new
segment, whereas loading a page requires
finding an unallocated page frame. The
l at t er problem is much less difficult t han
the former: whereas every unallocated page
frame is exactly the right size, not every
unallocated region may be large enough,
even though the sum of several such regions
may well be enough. (The question of find-
ing or creating unallocated regions will be
considered later.)
I t is possible to combine segmentation
and paging into one implementation, t hereby
accruing the advantages of both. Figure 7
shows t he essentials of such an addressing
mechanism. Each segment, being a small
linear name space in its own right, may be
described by its own page table. The sth
ent ry of the segment table contains :~ pair
(A, b) where A designates which page table
describes segment s and b is the limit for
segment s. The word address w is converted
to a pair (p, w t) as in paging, and p is used
to index page table A to find the frame
address pl containing page p. As before,
protection bits may be included in the seg-
ment table entry. As before, the segment and
page tables may be stored in memory, t he
addressing mechanism being appropriately
modified. As before, associative memory may
be used to speed up address formation;
indeed, the associative memory is essential
here, since each program-generated mem-
ory reference address incurs two table
references, and the program could run at one-
t hi rd speed without the associative memory.
(If the processor has a sufficiently rich
repertoire of register*to-register operations,
speed degradation would not be as bad as
one-third.)
We mentioned earlier t hat segmentation
and paging combined serve to achieve t he
objective of sharing or borrowing programs
(see the section on Segmentation .above).
Programmer X, who owns segment s, may
allow programmer Y to borrow s, and Y may
choose to call s by another name s'. Then
programmer X' s segment table will contain
(A, b) at ent ry s, and programmer Y's
segment table will contain (A, b) at e, nt r y s',
where A designates a single (shared) page
table describing the segment in question.
The details of implementation, as well as a
description of advantages and difficulties of
sharing segments, are adequately described
in [A5, B7].
Most addressing mechanisms use a single
register to implement the segment and word
registers shown separately in Figure 7. Typi-
cally the leftmost q bits of this register con-
tMn the segment name, and the rightmost r
bits contain t he word name; t hus there may
be as many as 2q segments and 2 ~ words
per segment. In these implementations t he
r word-bits serve as the program counter
(PC). Now suppose the program at t empt s to
increment the program counter (i.e. PC : =
Computing Surveys, Vol. 2, No. 3, September 1970
PC + 1) when its cont ent s are c( PC) =
2 * - 1; t he r esul t will be c( PC) = 0 and a
car r y f r om t he l ef t most pr ogr am count er
posi t i on. Some i mpl ement at i ons r equi r e
t ha t a segment ' s size l i mi t b sat i sfy 0 _<
b < 2 r, wher eupon t hi s car r y woul d t ri gger
an overfl ow faul t . Ot her i mpl ement at i ons
allow t he car r y t o pr opagat e i nt o t he seg-
ment field; t hus if c( PC) = 2 ~ - 1 in segment
s and t he oper at i on PC : = PC + 1 is
per f or med, t he resul t is c(PC) = 0 in seg-
ment s + 1 [R3].
STORAGE UTILIZATION
Our previ ous discussion has di r ect ed at t en-
t i on t o t he mechanisms of i mpl ement i ng
segment at i on, pagi ng, or bot h. A vi r t ual
memor y syst em, however, is mor e t ha n mer e
mechani sm; i t necessari l y i ncl udes t he
policies wher eby t he mechani sms are used.
We ment i oned earl i er t ha t policies fall i nt o
t hr ee classes:
1. Replacement policies. Det er mi ne whi ch
i nf or mat i on is t o be r emoved f r om memor y;
i.e. cr eat e unal l ocat ed regions of memor y.
2. Fetch policies. Det er mi ne when in-
f or mat i on is t o be l oaded; i.e. on demand or
in advance t hereof.
3. Placement policies. Det er mi ne where
i nf or mat i on is t o be pl aced; i.e. choose a
subset of some unal l ocat ed region.
Repl acement and f et ch policies use es-
sent i al l y t he same principles in bot h paged
and nonpaged syst ems, and pr esent t he
same degree of di ffi cul t y in ei t her case; we
t her ef or e defer discussion of t hese t opi cs
unt i l l at er. The pl acement pol i cy for pl aci ng
k pages in a pagi ng syst em is i n pri nci pl e
qui t e el ement ar y; use t he r epl acement pol i cy
t o free k pages. Pl acement policies f or non-
pagi ng syst ems are, however, consi derabl y
mor e i nvol ved. To i nvest i gat e why t hi s is
so, we consider a ve r y el ement ar y model f or
t h~ behavi or of a nonpaged memor y syst em.
Placement Policies
We suppose t ha t a l i near m-word memor y
is t o be used t o st ore each segment con-
t i guousl y (in t he manner of t he sect i on on
Segment at i on) . At cer t ai n moment s in t i me
transactions occur, which change t he con-
Virtual Memory * 165
ml
Fio. 8. Checkerboarding of memory
fi gurat i on of t he memor y. A t r ans act i on is
ei t her a r equest t o insert a new segment of
gi ven size, or t o delete some segment al r eady
present . We assume t ha t t he syst em is i n
equi l i bri um; i.e. t hat , over a long peri od of
t i me, t he number of i nsert i ons is t he same as
t he number of del et i ons f or segment s of each
size. ( For our purposes, t he f et ch pol i cy is
t he source of i nsert i on r equest s and t he
r epl acement pol i cy t he source of del et i on
request s. ) Af t er a l ong t i me, t he me mor y will
consist of segment s i nt er spaced wi t h holes
(unal l ocat ed regi ons); as suggest ed by Fi gur e
8, t he memor y has t he appear ance of bei ng
" checker boar ded. "
The pl acement al gori t hm, whi ch imple-
rnents t he pl acement pol i cy, makes use of
t wo t abl es: t he "hol e t abl e, " whi ch lists all
t he holes, and t he "s egment t abl e, " whi ch
al r eady exists f or use by t he addressi ng
mechani sm. An i nsert i on r equest f or seg-
ment s, whi ch al ways adds e nt r y s t o t he
segment t abl e, ma y i ncrease, l eave un-
changed, or decrease t he number of holes
dependi ng r espect i vel y on whet her s is
i nser t ed so as t o be sur r ounded by t wo boles,
a hol e and a segment , or t wo segment s.
The last possi bi l i t y occurs wi t h ve r y low
pr obabi l i t y and ma y be i gnored; and t he
first possi bi l i t y is usual l y pr ecl uded because
pl acement policies make i nsert i ons begi nni ng
at a boundar y of t he hole. A del et i on r equest
for segment s, which al ways r emoves e nt r y s
f r om t he segment t abl e, ma y decrease, l eave
unchanged, or i ncrease t he number of holes,
dependi ng r espect i vel y on whet her s is
sur r ounded by t wo holes, by a hol e and a
segment , or by t wo segment s. Bot h t he hol e
t abl e and t he segment t abl e must be modi fi ed
appr opr i at el y at each t r ansact i on.
We shall deri ve now t wo simple but im-
por t a nt rel at i onshi ps f or pl acement policies
havi ng t he pr oper t i es descri bed above. The
Comput i ng Surveys, Vol. 2, No. 3. Sept ember 1970
166 Peter J. Denmng
first is the "fi ft y percent rule" (due to Knut h
[K4]), which states t hat the average number
of holes is half t he average number of seg-
ments. The other is the "unused memory
rule, " which establishes a relation between
the difficulty of placing a segment and t he
amount of unused memory.
FIFTY PERCENT RVL~ [K4]. Suppose the
memory system described above is i n equilib-
ri um, having an average of n segments and h
holes, where n and h are large. Then h i s
approximately n / 2.
To establish this, we find the probability p
t hat an arbitrarily chosen segment has a hole
as right neighbor ( "r i ght " has meaning
according t o Figure 8). Over a segment' s
lifetime in memory, half the transactions
applying to t he memory region on its im-
mediate right are insertions, half are dele-
tions; thus p = . Therefore, the number of
segments with holes as right neighbors is
np = n/ 2, i.e. the number of holes is ap-
proximately n/ 2.
UNUSED MEMORY RULE. Suppose the
memory system described above is i n equilib-
ri um, and let f be the fraction of memory
occupied by holes. Suppose furt her that the
average segment size is So and that the average
hole size is at least kSo for some k > O. Then
f >_ k/ ( k + 2).
To establish this result for an m-word
memory we note t hat , by t he fifty percent
rule, there are n / 2 holes in memory; since
each segment occupies an average space of
size so, the amount of space occupied by
holes is m -- nso, and t he average space per
hole (hole size) is x = (m -- nso)/h =
2(m - nso)/n. But we assume x ~ ks0,
which implies
(hi m)so < 2/(k + 2).
Then
f = ( m- - n s o ) / m = 1- (n/m)so
_> ~ - 2/ (k + 2) = k/ (k + 2).
In other words, if we wish to limit place-
ment algorithm overhead by maintaining
large holes, we must be prepared t o " pa y"
for this limitation by "wast i ng" a fraction f
of memory. This is not quite as serious as it
might seem, for simulation experiments
[K4] show t hat there is a large variance in
hole sizes, and it is often possible to make f
as small as 10 percent (i.e. k approximately
). Even so, it is not possible to reduce f t o
zero.
Of t he many placement algorithms having
the properties described above, there are two
of special interest. The first is appealing
because it makes best use of holes, and the
second is appealing because it is simple t o
implement. Assume there are h holes of sizes
xl , x2, . , xh, and an insertion request of
size s arrives.
1. Best fit. The hole table lists holes in
order of increasing size (i.e. xl ~_ x2 _< . . .
_< xh). Fi nd the smallest i such t hat s _< xi .
2. Fi rst fit. The hole table lists holes in
order of increasing initial address. Fi nd the
smallest i such t hat s < xl . (After a long
time, small holes would t end to accumulate
at the head of the hole list, t hereby increas-
ing t he search time. To prevent this, the
hole table is implemented as a circular list
with a "st ar t poi nt er"; each search advances
the pointer and begins searching with the
designated hole.)
Knut h [K4] reports detailed simulation
experiments on these and other placement
policies. He finds t hat the first-fit algorithm
is t he most efficient of a large class of al-
gorithms, including the best-fit. He finds also
t hat the memory size must be at least ten
times the average segment size for efficient
operation. Similar conclusions are also
reported by Collins [C6].
Knut h reports also on another algorithm
which he found slightly better t han first-fit
but which, being not in the class of placement
policies described above, does not follow the
fifty percent rule and the unused memory
rule. This policy is called the "buddy sys-
t em. " Its dynamic properties have not yet
been completely deduced [K4].
3. Buddy system. Assume t hat t he re-
quest size is s = 2 ~ for some i ~ ,~. This
policy maintains k hole-lists, one for each
size hole, 21, 2 ~, . . . , 2 k. A hole may be re-
moved from t he (i + 1)-list by splitting it in
half, thereby creating a pair of "buddi es" of
Computing Surveys, Vol. 2, No. 3, September 1970
sizes 2 i, whi ch are ent er ed in t he /-l i st ;
conversel y, a pai r of buddi es ma y be r emoved
f r om t he / - l i s t , coalesced, and t he new hol e
ent er ed in t he (i + 1)-list. To find a hol e of
size 2 ~, we appl y t hi s pr ocedur e r ecur si vel y:
procedure gethole(i)
begi n i f i ~ k + 1 t hen report failure;
i f /- l i st empty t hen
begi n hole : = gethole(i + 1);
split hole into buddies;
place buddies in/-list;
end
gethole := first hole in/-list;
end
Overflow and Compaction
The unus ed- memor y rul e tells us t hat , in
equi l i bri um, we must t ol er at e a significant
loss of memor y. I n t er ms of Fi gur e 8, t he
memor y has become so checker boar ded t ha t
t her e are ma ny small holes, col l ect i vel y
r epr esent i ng a subst ant i al space. I ndeed, i t
is possible t hat , when we scan t he hol e sizes
x~, x~, . , xh f or a r equest of size s, we find
s > x~, 1 < i < h (i.e. t he r equest can-
not be satisfied) even t hough s < ~]~=1 x~
(i.e. t her e is enough space di st r i but ed among
t he holes). Wha t can be done about t hi s?
The sol ut i on usual l y pr oposed calls f or
"compact i ng me mor y, " i.e. movi ng seg-
ment s ar ound unt i l several holes have been
coalesced i nt o a single hol e large enough t o
accommodat e t he gi ven r equest . Kn u t h [K4]
r epor t s t ha t si mul at i on exper i ment s showed
t hat , when t he first-fit al gor i t hm began t o
encount er overflow, memor y was near l y full
anyway; t hus compact i ng i t woul d pr ovi de
at best margi nal benefit. I n ot her words, a
good pl acement pol i cy t ends t o obvi at e t he
need f or a compact i ng pol i cy.
A somewhat di fferent poi nt of vi ew can be
adopt ed regardi ng t he role of memor y com-
pact i on. I ns t ead of using a sophi st i cat ed hol e
sel ect i on pol i cy and no compact i on, we ma y
use a sophi st i cat ed compact i on pol i cy and
n o hol e selection. Just as over head in mai n-
t ai ni ng t he hol e list pr evi ousl y l i mi t ed our
abi l i t y t o use memor y fully, so t he over head
in r unni ng a compact i on pol i cy limits our
abi l i t y t o use memor y fully. To show t hi s,
we consi der t he compact i on scheme sug-
Virtual Memory 167
/
I~ wm(#- #) ..L mf .~
V T
HOL
Fio. 9. Configuration of memory after compaction
gest ed in Fi gur e 9. At cer t ai n moment s in
t i me - - " c ompa c t i on i ni t i a t i ons " - - c omput e r
oper at i on is suspended and all segment s are
moved t oget her at t he low end of memor y,
cr eat i ng one large hol e at t he hi gh end of
memor y. Each i nsert i on r equest is pl aced at
t he low end of t he hole, t her eby movi ng t he
boundar y r i ght war d; when t he bounda r y
reaches t he hi gh end of memor y, t he next
compact i on i ni t i at i on occurs.
COMPACTION RESULT. Suppose the mem-
ory system described above is in equilibrium,
a fraction f of the memory being unused;
suppose that each segment is referenced an
average r times before being deleted, and that
the average segment size is So. Then the fraction
F of the time system expends on compaction
satisfies F ~ (1 -- f ) / [1 -- f + (f/2)(r/so)].
To est abl i sh t hi s resul t , obser ve t ha t a refer-
ence occurs t o some segment in me mor y each
t i me uni t , and t ha t one segment is del et ed
ever y r references. Because t he s ys t em is i n
equi l i bri um, a new segment mus t be i nser t ed
ever y r references; t her ef or e t he r at e of t he
boundar y' s movement is so/r words per uni t
t i me. The syst em' s oper at i on t i me to is t hen
t he t i me r equi r ed for t he bounda r y t o cross
t he hole, i.e. to = f mr/ so. The compact i on
oper at i on re~luires t wo me mor y r ef er ences - -
a f et ch and a s t or e- - pl us over head f or each
of t he (1 - f ) m words t o be moved, i.e. t he
compact i on t i me t~ is at l east 2(1 -- f)m.
The f r act i on F of t he t i me spent compact i ng
is F = 1 - to/(to + t~), whi ch r educes t o t he
expressi on given.
Fi gur e 10 shows a pl ot of F versus f , f r om
whi ch i t is evi dent t hat , if we are t o avoi d
expendi ng significant amount s of t i me com-
Co mp u t i n g Sur ve ys , Vol. 2, No. 3, Se p t e mb e r 1970
; Z!
168 Peter J. Denni ng
F
o. e .-~, -, I
( 1 4
O. 4
0 . 2
0 f
o e . z 0.4 0.6 o.e Lo
F ro. 10. Inefficiency of compaction
pacting, we must tolerate a significant waste
of memory. Because of the relative slowness
of compaction compared to searching a well-
organized hole list, the former tends to be
less efficient t han the latter, and compaction
is not often used.
In summary, nonpaged memory requires
an "i nvest ment , " i.e. a certain amount of
unused memory and overhead in placement
policies, for efficient operation. Some sys-
tems, not abl y the Burroughs B5000 series
[R3] and certain CDC 6600 installations
[B1], have chosen to make this investment;
but most have elected to use paged memory,
which can be fully utilized by pages at all
times. Many of the techniques discussed in
this section have been used with great suc-
cess in applications of a less general purpose
nature, particularly in list-processing sys-
tems [BS, C4, K4].
,)
Fro. 11. Probability of external fragmentation
Fragmentation
Our discussion in the previous section
unveiled a problem of some importance in
virtual memory systems; storage fragment a-
tion, the inability to assign physical locations
to virtual addresses t hat contain informa-
tion.
There are three major types of storage
fragmentation. The first is external frag-
mentation [R2], which occurs in nonpaged
memories when checkerboarding becomes so
pronounced t hat every hole is too small to
be used. (More precisely, external frag-
mentation occurs for segments of size s with
probability E(s), the probability t hat
s > max{xl}, where {xi} are the hole sizes.
E(s) follows the curve suggested in :Figure
11.) The second is internal fragment at i on
[R2], which results in paged memories be-
cause storage requests must be rounded up
to an integral number of pages, the last part
of the last page being wasted (Figure 12).
(More precisely, if z is the page size and s a
segment size, t hen s is assigned to k pages,
where (k -- 1)z < s < kz; t hen kz - s
words are wasted inside the last page.) The
t hi rd is table fragmentation, which occurs in
both paged and nonpaged memories because
physical locations are occupied by mapping
tables and are therefore unavailable for
assignment to virtual addresses.
Randell [R2] reports simulation experi-
ments showing t hat fragmentation raay be
serious, and t hat internal fragmentation is
more troublesome t han external. His experi-
ments rely on three assumptions: (1) each
segment is entirely present or entirely
missing from memory, (2) each segment
begins at a new page boundary, and (3)
segments are inserted or deleted one at a
time. Many systems violate (1), there being
some nonzero probability t hat a segment' s
final page is missing. Many systems violate
(2) and (3) by providing facilities theft allow
I ~ I 2 k
L I - l a o
v !
F~a. 12. Internal fragmentation
Computing Surveys, Vol. 2, No. 3, September 1970
ma ny smal l segment s t o be combi ned i nt o
one l arge cont i guous region of address space
(e.g. a "r el ocat abl e l oader " for vi r t ual
memor y, or a file syst em separ at e f r om
vi r t ual memor y) . Thus f r agment at i on is not
as serious in pr act i ce as i t coul d be, but t hen
agai n i t cannot be ignored.
Page Size
Two f act or s pr i mar i l y influence t he choice
of page size: f r agment at i on, and efficiency
of page- t r anspor t operat i ons.
Ther e is a page size opt i mal in t he sense
t ha t st orage losses are mi ni mi zed. As t he
page size increases, so increases t he likeli-
hood of wast e wi t hi n a segment ' s l ast page.
As t he page size decreases, so increases t he
size of a segment ' s page t abl e. Somewher e
bet ween t he ext r emes of t oo large and t oo
smal l is a page size t ha t minimizes t he t ot al
space l ost bot h t o i nt er nal f r agment at i on
and t o t abl e f r agment at i on.
OPTIMAL PAGE SI ZE RESULT. Let z be
the page size and so the average segment si ze;
suppose el i s the cost of l osi ng a me mor y word
to table f r agment at i on and c2 the cost of l osi ng
a me mor y word to i nt ernal f r agment at i on,
and let c = c~/c~ . I f z << so, the opt i mal page
si ze Zo i s appr oxi mat el y (2cs0) t.
To est abl i sh t hi s resul t , suppose segment
size s is a r andom var i abl e wi t h expect at i on
E[s] = so. A segment may be expect ed t o
occupy appr oxi mat el y so/z pages, each bei ng
descri bed by one page t abl e word; t he page
t abl e cost f or t hi s segment is t her ef or e ap-
pr oxi mat el y ClSo/Z. I f z << so, t he expect ed
loss inside t he l ast page is appr oxi mat el y
z/ 2; t he i nt er nal f r agment at i on cost for t hi s
segment is t her ef or e appr oxi mat el y c~z/2.
The t ot al expect ed cost for f r agment at i on is
t hen
Ei C I z] = (So/Z)C~ + (z/ 2)c~ .
I f we set d E[ C] z ] / d z = 0 and solve f or z,
we obt ai n t he expressi on gi ven for z0.
These resul t s pr esume t ha t each segment
begins on a page bounda r y (as suggest ed by
Fi gur e 12), and t ha t bot h t he segment and
i t s page t abl e are ent i r el y pr esent in memor y.
Ma n y vi r t ual memor y comput er s pr ovi de
mechani sms f or l oadi ng or rel ocat i ng a col-
l ect i on of segment s cont i guousl y in address
Vi r t ual Me mo r y 169
space, in whi ch t he i nt er nal f r agment at i on
will occur onl y in t he l ast page of t he l ast
segment in a such collection. I f t her e are k
segment s in such a col l ect i on on t he average,
t hen t he foregoi ng resul t s ar e modi fi ed by
repl aci ng so by ks0, whence zo = (2ckso) ~.
These resul t s are by no means new. I n
fact , t he pr obl em of choosi ng page size t o
mi ni mi ze f r agment at i on is i dent i cal t o t ha t
of choosing bl ock size i n var i abl e l engt h
buffers t o mi ni mi ze space lost t o i nt er nal
f r agment at i on and t o chai ni ng i nf or mat i on.
Wol man [W7] has st udi ed t hi s issue in some
det ai l ; he gives a det ai l ed account of t he
accur acy of t he appr oxi mat i on z0 ~= (2s0) .
Wha t mi ght be a t ypi cal val ue f or z0?
The avai l abl e dat a on segment size [B2]
suggests t ha t so g 1000 words i n most cases;
t aki ng t hi s and c = 1, we find z0 _~ 45 words.
Thi s is r at her st ar t l i ng when we consi der
t ha t pages of 500-1000 words are commonl y
used.
When we consi der t he ot her f a c t o r - -
efficiency of page- t r anspor t oper at i ons - - we
di scover t he mot i vat i on f or usi ng a l arge
page size. Each page- t r anspor t oper at i on
t akes one t r ans por t t i me T (see t he sect i on
on Basi c Sys t em Har dwar e above) t o be
compl et ed. The following expressi ons for T
on t ypi cal devi ces are l ower bounds because
in deri vi ng t hem, we have i gnor ed queuei ng
del ays and processor over head expended on
name conversi on and auxi l i ar y me mor y
cont rol .
1. Dr ums . To obt ai n a page f r om a
dr um, one must wai t an aver age of hal f a
dr um r evol ut i on t i me t~ f or t he i ni t i al wor d
of t he desi red page t o r ot a t e i nt o posi t i on.
I f t her e are w words on t he ci r cumf er ence
of t he dr um, t he page t r ansf er t i me tt is
t rz/ w. Ther ef or e
T = t r / 2 + t t = t , ( 1/ 2 + z/ w) .
Typi cal l y, t~ = 16 msec and w = 4000
words.
2. Di s ks (movi ng ar m). A di sk access is
j ust like a dr um access except t her e is an
addi t i onal "seek t i me " t~ r equi r ed t o move
t he arms i nt o posi t i on. Ther ef or e
T = t~ + t r / 2 + t t = t , + t ~(1/2 + z/ w) .
Computing Surveys, Vo|. 2, No. 3, September 1970
170 Peter J. Denni ng
T ( pI J
,o'
,o'
IO
c
i I0 I00 0
F z o. 13. Lo we r b o u n d t r a n s p o r t t i me s
z
1.0 LCS
0.8 ~ ECS
0.6. / / DRUM
0.4.
0 z
I I0 I00 I000
FIG. 14. Upper bound transport efficieneies
Typi cal l y, t8 = 100 msec, t~ = 30 msec,
and w = 4000 words.
3. Large capacity storage (LCS). Thi s is
not hi ng mor e t ha n a slow-speed core mem-
ory. I f i t s cycl e t i me is t c, t hen
T = t~ = ttz.
Typi cal l y, t~ = 10 psec.
4. Extended core storage (ECS). Thi s is a
f or m of core me mor y wi t h special t ransmi s-
sion facilities; af t er an i ni t i al "access t i me "
t a, i t del i vers v words per mai n memor y
cycle. Ther ef or e
T = t~, + t , . = ta + (Z/v)tc.
Typi cal l y, t~ = 3 psec, tc = 1 psec, and v =
10 words.
Fi gur e 13 shows t hese f our l ower bound
t r ans por t t i me expressi ons pl ot t ed for vari ous
val ues of z. Not e t he several orders of magni -
t ude differences at smal l page sizes. Fi gur e
14 shows t he cor r espondi ng upper bound
efficiencies e = t t / T pl ot t ed f or var i ous
val ues of z. I t is i mmedi at el y appar ent f r om
t hese figures t ha t movi ng- ar m disks shoul d
never be used, nei t her for pagi ng appl i cat i ons
nor f or a ny ot her heavy- t r af f i c auxi l i ar y
memor y appl i cat i ons [D3]. I t is also appar ent
t ha t dr ums shoul d be used wi t h care [C2,
D3]; and t ha t if dr ums are used, a page size
of at l east 500 words is desi rabl e. ']?his is
why most pagi ng syst ems use dr ums i nst ead
of movi ng- ar m disks for auxi l i ar y st orage,
why page sizes of 500-1000 words are com-
mon in t hese syst ems, and why some syst ems
have been exper i ment i ng wi t h LCS [F1],
ECS [F5], and ot her [L2] auxi l i ar y st ores.
I t is equal l y appar ent t ha t t her e is a gr eat
di scr epancy bet ween t he page size f or maxi -
mi zi ng st orage ut i l i zat i on and t he page size
f or maxi mi zi ng page- t r anspor t ef f i ci ency- -
about t wo orders of magni t ude di scr epancy.
I t is easy t o see t ha t t he poor per f or mance of
some of t hese syst ems [K6] is at l east par -
t i al l y at t r i but abl e t o t hi s fact or.
I t is somet i mes ar gued t ha t anot her f act or
i nhi bi t i ng smal l page sizes is t he addi t i onal
har dwar e cost t o accommodat e t he l arger
number of pages. Wher eas t hi s har dwar e cost
is an i ni t i al one- shot i nvest ment , t he in-
creased st or age ut i l i zat i on pr ovi des a con-
t i nui ng l ong- t er m payoff, and t he ext r a
har dwar e is pr obabl y wort hwhi l e. The
cache st ore on t he I BM 360/85 is an ex-
ampl e of a syst em where t hi s i nves t ment
has been made, wi t h appar ent l y good effect.
One appr oach t o const r uct i ng a syst em in
which a page size z0 is feasible woul d be t o
use a much f ast er device, such as LCS or
ECS, t o handl e t he traffic of pages in and
qut of mai n memor y. Some syst ems have
adopt ed t hi s appr oach IF1, F5, L2].
Computing Surveys, Vol. 2, No. 3, September 1970
Anot her a ppr oa c h- - " pa r t i t i one d segmen-
t a t i o n " - - h a s been suggest ed by Randel l
[R2]. I t effects a compromi se bet ween t he
l arge page size r equi r ed for t r ans por t effi-
ci ency f r om r ot at i ng auxi l i ary devi ces and
t he small page size r equi r ed for good st orage
ut i l i zat i on. We shall descri be a slight var i ant
t o Randel l ' s scheme. The memor y syst em
uses t wo page sizes: a " maj or page" whose
size is chosen t o make t r anspor t s efficient,
and a "mi nor page" whose size is chosen
close t o z0. Suppose t he maj or page size is
Z and t he mi nor page size z, where Z is a
mul t i pl e of z. A segment of size s is assigned
a " he a d" consisting of K maj or pages such
t ha t ZK < s < Z( K + 1), and a " t a i l "
consisting of k mi nor pages such t ha t
zk < s - ZK < z(k-4- 1 ) , a n d Kq - k _ > 1.
I nt er nal f r agment at i on t hus occurs onl y
wi t hi n t he last mi nor page. An address
t r ansl at i on mechani sm t ha t i mpl ement s
par t i t i oned segment at i on is shown in Fi gur e
15. A maj or dr awback t o t hi s scheme is
t hat , t o oper at e effect i vel y, segment s must
be l arge enough so t ha t t he y consist most l y
of maj or pages. Avai l abl e dat a [B2] suggests
t ha t this need not be t he case.
Compression Factor
Dur i ng any gi ven run, cer t ai n sections of
a pr ogr am' s code will never be r ef er enced
because condi t i onal br anch i nst r uct i ons will
have unf avor abl e out comes. I n ot her words,
an n- wor d pr ogr am will, on a gi ven run,
have occasion t o reference onl y n' < n of
i t s words, and n' - n addresses will have
been unreferenced. These n' - n unrefer-
eneed words are said t o be superfl uous [K5].
St or age losses due t o l oadi ng superfl uous
words i nt o mai n memor y are less serious i n
paged memori es using small page sizes
because, for small page size, unr ef er enced
bl ocks of code will t end t o be i sol at ed on
t hei r own pages, whi ch need never be
br ought i nt o memor y. Bel ady' s si mul at i ons
[B3] and O' Nei l l ' s dat a [02] confi rm t hi s.
The more are superfl uous words i sol at ed
on t hei r own pages, t he less space will a
pr ogr am requi re, and t he mor e "compr es-
si bl e" will i t be. For page size z and a gi ven
r un of t he pr ogr am, define t he compressi on
fact or c(z) t o be t he r at i o of t he number of
Vi rt ual Me n wr y 171
SEGMENT PAGE
TABLE ST T ABL E p' r a
OPERAT I ON :
(s, w) Loaded i nt o segment and word regi sters
if sth entry of ST blank, missing-segment fault
if w > b, overflow fault
p := [w/Z]
p" :-=O
if pth entry of FT, marked by *,
p" := [(w - ZK)/z.]
if (p p")-th entry of PT~ blank, missing-page
fault
vo' := R, (w -- ZK)
(p' w' ) loaded into MA
Fro. 15. Partitioned segmentation
referenced pages t o t he t ot al number of
pages. Th a t c(z) -- x implies t ha t at l east a
f r act i on 1 - x of a pr ogr am' s wor ds ar e
superfluous, or conver sel y t ha t x is t he
maxi mum r el at i ve amount of memor y space
a pr ogr am needs on a gi ven run. Not e t ha t
c(n) = 1 and c(1) = n ' / n . Accordi ng t o t he
da t a pr esent ed by Bel ady [B3] and O' Nei l l
[02], t he compressi on f act or is appr oxi mat ed
by t he expressi on
c(z) = a -~ b log2z, 25 < z _< 2",
where a .~ 0 and b > 0. The da t a suggests
t he following pr oper t i es of c(z):
1. Hal vi ng t he page size t ends t o decrease
t he compressi on f act or by 10 t o 15 per cent ;
t hus 0.10 ~ b ~ 0.15 [B3].
2. For smal l z, 1 < z < 25, t he expressi on
a + b log2 z is a l ower bound on c(z), and in
par t i cul ar c(1) = n ' / n _> a. Ext r apol at i ng
t he dat a, a i n t he r ange 0.1 ~ a < 0.4
appear t ypi cal .
3. For page sizes z ~ 29, c(z) > 0.8
appear t ypi cal .
These resul t s are significant. Th e y r eveal
a f r equent l y over l ooked pot ent i al advant age
of vi r t ual memor y: smal l page sizes per mi t a
great deal of compressi on wi t hout loss of
efficiency. Smal l page sizes will yi el d signifi-
cant i mpr ovement s in st or age ut i l i zat i on,
Colnputing Surveys, Vol. 2, No. 3, September 1970
]
172 Peter J. Denning
TABLE I. COMPARISON OF PAGED AND NONPAGED MEMORY
Factor .Paged Nonpaged
Segmented name space
Number of memory accesses per pro-
gram reference:
1. With paging
2. With segmentation
3. With both
4. With associative memory mapping
Replacement policy
Fetch policy
Placement policy
Memory compaction
External fragmentation
Internal fragmentation
Table fragmentation
Compression factor
Feasible Feasible
2
3
Required
Usually demand
Required, but simple
Not required
None
Yes, but can be controlled by
proper choice of page size
Yes
Can be much less than 1 with
small page sizes
2
Required
Usually demand
Required, but complicated
Optional; of marginal value
Yes; controlled by placement
policy and memory size at
least ten times average
segment size
None
Yes
Usually 1
over and above t hose gai ned by mi ni mi zi ng
f r agment at i on. Nonpaged memor y syst ems
(or paged syst ems wi t h l arge page sizes)
cannot enj oy t hi s benefi t .
COMPARISON OF PAGED AND NONPAGED
MEMORIES
As we have discussed, the various imple-
mentations of virtual memory fall into two
classes: paged and nonpaged. We have
di scussed a gr eat number of fact s per t ai ni ng
t o each. Tabl e I summari zes t hese fact s and
compar es t he t wo met hods.
Accordi ng t o Tabl e I , pagi ng is superi or
t o nonpagi ng in all respect s save suscept i -
bi l i t y t o i nt er nal f r agment at i on; hut i nt er nal
f r agment at i on can be cont r ol l ed by pr oper
choice of page size. Not l i st ed i n t he t abl e
is an aspect of paged me mor y t ha t makes
i t s i mpl ement at i on mor e el egant and much
" cl eaner " t ha n i mpl ement at i ons of non-
paged memor y: its " uni f or m" t r e a t me nt of
f nemor y. Wher eas pagi ng r egar ds mai n
memor y si mpl y as a pool of anonymous
bl ocks of st orage, segment at i on regards i t
as a pat chwor k of segment s and holes of
var i ous sizes. The same s t at ement hol ds
f or auxi l i ar y memor y. Ther ef or e (fixed
l engt h) page t r ans por t s are much si mpl er
t o manage t ha n (vari abl e l engt h) segment
t r anspor t s. The di ffi cul t y of t r ans por t i ng
var i abl e l engt h segment s is compounded
by over head in wat chi ng out for t he specific
segment l engt h in or der not t o over r un
buffers. I t is no surpri se t ha t some f or m
of pagi ng is used in al most all vi r t ual mem-
ories.
Computing Surveys, "~ol. 2, No. 3, September 1970
DEMAND PAGING
Because paging is so commonly used and so
frequently discussed in the literature, t he
remainder of our discussions center around
this topic. Demand paging, t he simplest
form, is t he most widely used. Demand
paging has- - unf ai r l y- - been subj ect ed t o
widely publicized criticism [F2, F4, K6, R3],
before anyone has had enough experience
to eval uat e it properly.
In order to avoid maintaining a large
number of lightly used resources, time-
sharing and multiprogramming systems
at t empt t o increase t he load factors on
resources by sharing them. To do this, time
is partitioned into disjoint intervals, each
program being allocated resources during
certain intervals but not during others.
(This is sometimes called resource multi-
plexing.) These intervals are defined either
naturally, by the alternation between
running states and i nput -out put waiting
states of processing, or artificially, by time
quanta and preemption. The latter method is
used primarily in time-sharing systems,
where response-time deadlines must be
satisfied. We restrict attention to this case
t hroughout this section.
At t he beginning of its allotted time
quant a, a program' s working information
must be loaded into main memory. Older
time-sharing systems employed swapping
t o do this, i.e. t hey would t r anspor t ' a pro-
gram' s working information as a contiguous
unit into memory j ust before each time
quant um began, and o u t of memory j ust
after each time quant um ended. Demand
paging systems transport j ust one page (that
containing the next instruction to be exe-
cuted) into memory j ust before a program' s
time quant um begins, and "page i n" addi-
tional pages as the program demands t hem;
at time quant um end, no immediate action
will be t aken to remove a program' s pages
from memory, t hat being left up to t he
replacement policy.
One occasionally hears proposals t o t he
effect t hat paging systems could be improved
markedly if swapping were used t o load
(unload) a program' s working information
Virtual Memory 173
at t he beginning (end) of a time quant um,
and demand paging were used within a time
quant um. We shall show t hat swapping is
at best of marginal value in systems using
either a nonmoving auxiliary store or a
specially organized drum, t he paging drum.
Prepaging, however, may have some value
when properly managed from a paging drum.
Paging Drum
We poi nted out i n the secti on on Page
Size above t hat among all rotating or moving
auxiliary stores, only drums (or drumlike
stores [A1]) may be suitable for handling
page traffic through main memory. Even
then, a particular drum organization is
required for efficient operation. A paging
drum [A1, C2, D3, W1] consists of a drum
memory together with hardware (or soft-
ware) implementing an optimal scheduling
policy. As shown in Figure 16, t he drum
surface is laid out into equal areas, each
capable of storing one page ; each such "dr um
page" is identified by its "sector address"
i and its "field address" j . Each field is
equipped with a set of read-write heads. As
shown in Figure 17, t he scheduler sorts
incoming requests into s separate "sector
queues" according as which sectors are
requested. Within a given sector queue,
service is in order of arrival (i.e. "first-come-
first-served"). The rot ary switch arm re-
volves synchronously with t he drum, point-
ing t o queue i whenever sector i is under
the read-write heads. Suppose a read (write)
request for drum page (i, j ) is at the head
of sector queue i. Just as t he switch arm
SECTOR | FIEi D j DROM .PAGE O,j)
READ-WRITE
HEADS
FiG. 16. Layout of paging drum
Comput i ng Surveys, Vol, 2, No. 3, Sept ember 1970
174 Peter J. Denning
am~n Ho n t ~$r $
/
x /
'N
SECTOR QUEUE
Fio. 17. Paging drum queue organization
reaches sector queue i, t he heads for field j
are set to read (write) status and connected
to the drum channel. Then transmission
begins.
Paging drums are sometimes known as
"slotted drums" or "shortest-access-time-
first" drums. Some manufacturers market
drumlike "paging disks," which are fixed-
head disks with one head per track These
are equivalent to paging drums.
The paging drum stands in contrast to its
historical predecessor, t he "first-come-first-
serve" (FCFS) drum, which collects all
incoming requests into a single, order-of-
arrival queue. To compare these, we imagine
two systems: System P is a paging drum,
and System F an FCFS drum. I n both
systems, the drum revolution time is t~ and
the number of sectors is s. Since most drum
allocation policies do not at t empt to group
contiguous pages of a given program on
contiguous sectors, we may assume t hat
each request selects a sector at random
[A1, C2, D3, W1]. The "dr um l oad" L is
the number of requests waiting in the drum
queue(s).
DRUM EFFICIENCY RESULT. Let ep(L)
denote the expected e~ciency of System P
and ep(L) that of System F, when the drum
load is held fixed at L. Then
ep(L) = (L + 1)/(s --k L -k 1),
L> i .
eF ( i ) = 2/ ( s -b 2),
Consider System P. The expression for
ee(L) is an approximation derived as follows.
Let to, h , ' " , t k , . . , be a sequence of
time instants at which requests csmplete
service and depart from the drum system.
(Since L is assumed fixed, a new request is
added to t he drum system at each time
tk .) Then xk = tk - tk-1 denotes the service
time of the kth request, Since the requested
sector positions are statistically independent
and L is fixed, the service times x, have a
common distribution with expectation E[x].
Now, E[x] can be written E[x] = t -~- E[r],
where t = t~/s i s ' a transmission time and
E[r] an expected rotational delay. To ap-
proximate E[r], we imagine a circle with
circumference tr having L -b 1 points dis-
t ri but ed randomly about its perimeter; one
of these points represents the drum position
at t he moment a request departs, and t he
remaining L points represent the positions
of t he requested sectors. The expected
distance between two of these points is
t r/(L q- 1) = E[r]. The efficiency is ep(L) =
t/E[x] = t /(t + E[r]), which reduces to the
expression given. In System F, each request
in t he queue must complete its service
before the next may begin, and each selects
its sector randomly. Therefore eF(L) is
independent of L, and indeed el (L) =
e , 0 ) = e~(1 ) = 2/ ( s + 2).
Several facts follow from this result. (1)
For small page sizes (large s) the efficiency
e~ is always small. (2) For any page size
there are always values of L t hat make ee
close to 1. (3) Whereas eF is constant,
ee(L -b 1) > ee(L); in other words, t he
paging drum is "self-regulating," becoming
more efficient under heavier loads. (4) For
L > 1 and s ~_ 1, ee(L) > eF(L).
As one would suspect, the paging drum
(System P) gives smaller transport times
t han t he less efficient FCFS drum (System
F).
DRUM TRANSPORT TIME RESULT. Sup-
Computing Surveys, Vol. 2, No. 3, September 1970
pose a page request arrives when the drum
load is L. The time each system delays this
request is
Tp = t ~(L/s + (s + 2)/2s),
L~ O.
T~ = t , (L + 1)(s -4- 2)/2s,
The incoming request will be known as
the "t agged" request. I n System P, the
tagged request enters a sector queue whose
expected length is L' = L/ s . Before com-
pleting service the tagged request experi-
ences the following additive delays: L/ 2 for
the drum to begin serving the first request
in the queue; L' t , for the drum to begin
serving the tagged request; and t~/s for its
own transmission. Thus T~ = t~(L' -4-
+ 1/s). In System F, the tagged request
joins the single queue with L requests ahead
of it. There are now L + 1 requests in the
t 1 queue, each requiring time r(~ + 1/S) to
complete.
From these two results we see t hat , under
normal drum loads (L > 0), ee > e~ and
T~ < TF, with the greatest differences
occurring at heavy loads. For these reasons,
paging systems using FCFS drums may
experience severe loss of efficiency.
Cost
To evaluate the "cost " of demand paging,
two concepts are useful: "space-time prod-
uct " and "working set . " Suppose a program
occupies re(t) pages of memory at time t;
t he space-time product of memory usage
across an interval (t~, t2) is defined to be
C(t~,h) = re(t ) dt.
1
Since memory usage charges are usually
based both on the extent and duration of
memory usage, C(t~, t~) relates to the actual
dollar cost of using memory, and is often
termed "cost ." Space-time cost has become
an i mport ant aid in determining the efficacy
of memory allocation strategies [B5, B6, D5,
D9, F1, L1, P1, R3]. The working set of a
program at a given time is the smallest
collection of its pages t hat must reside in
memory to assure some level of efficiency
(~ more precise definition will be given later)
[D4, D5].
Virtual Memory . 175
Let Cd(A) denote the space-time cost
of loading a working set into memory under
demand paging from auxiliary memory A,
and Ca(A) the cost of loading a working set
into memory under swapping from auxiliary
memory A. We shall establish f()ur asser-
tions:
Under demand paging, t he paging drum
costs significantly less t han the FCFS drum
(i.e. Cd(F) -- Cd(P) is large).
With nonmoving auxiliary storage
(e.g. A is LCS or ECS), demand paging never
costs more t han swapping (i.e. Ca(A) _<
Cs(A)).
The combined swapping and demand
paging strategy is at best of questionable
value when compared to "pur e" demand
paging with a paging drum.
Unless predictions can be made with
little error, prepaging, even from a paging
drum, may not be advantageous.
These assertions are considered in the
following paragraphs. Assume t hat a working
set of size w is to be loaded, t hat a single
transport operation requires processor time
to, and t hat t he drum has revolution time
tr and s sectors.
The cost Cd(A) is determined as follows.
Suppose k - 1 of the w pages have already
been loaded and a fault for the kth page
occurs; we must reserve one more page of
memory and stop the program for the kth
transport time Tk. Since there is no correla-
tion between the order of page calls and
their order of storage on the drum, Tk = T
for 1 < k <: w. Thus
Cd(A) = 5~ kTk = kT
~1 (i )
= T(w(zo + 1)/2).
Now if A is the paging drum system P (see
the section on Paging Drum above), t hen
T = to + Te. Similarly, T = to + TF for
the FCFS drum system F. Appl yi ng the
Drum ~Fransport Time Result for load L,
Cd(F) - Cd(P) = (W(W. + 1)/2) t~(L/2).
As long as L >_ 1 (the usual case) t he cost
difference grows as the square of t he working
set size. This establishes the first assertion.
Computing Surveys, Vol. 2, No. 3, September 197~)
:i
176 Peter J. Denni ng
The cost Ca(A) is det er mi ned as follows.
We reserve w pages of memor y, t he n t r ans-
por t t he ent i r e worki ng set as a uni t i n a
t r ans por t t i me T t. Thus
Ca(A) = wT' . (ii)
I f A is ECS wi t h access t i me t~ (see t he
sect i on on Page Size above) and page t r ans-
mission t i me t t , t he n
T = t 0 + t~ + t~,
T' = to + t~ "b wtt.
Subst i t ut i ng t hese vMues i nt o (i) and (ii)
r espect i vel y, we find
C~(A) - Cd(A)
= (w(w - 1) / 2) ( t t - to - t, ).
Thi s expressi on is posi t i ve if tt ~_ to -q- t~,
whi ch nor mal l y is t he case. I f A is LCS,
t hen t~ = 0, and t he same concl usi on follows.
Thi s est abl i shes t he second assert i on.
A "swappi ng d r u m" is an FCFS dr um F
f or whi ch t he syst em guar ant ees t ha t each
worki ng set ' s pages are st or ed on cont i guous
sectors. Suppose a r equest f or a wor ki ng
set of w pages arri ves when L ot her r equest s
of sizes ~ , , vL ar e in t he dr um queue;
t he swappi ng dr um t r ans por t t i me is gi ven
by
1
( The ar gument t o der i ve T' is anal ogous t o
t ha t f or der i vi ng Tp .) We are i nt er est ed
i n compar i ng
Cd(P) = (w(w + 1)/2)(t o + Te),
C, (F) = wT' .
We shal l i gnore to since to << t~. Consi der
t wo ext r emes of t he dr um l oad v~, . . . , vL
At t he one, each v~ is a r equest t o swap in a
ful l worki ng set ; t aki ng w as t he aver age
worki ng set size and each v~ = w, we find
( af t er some al gebra) t ha t f or all w > 0,
C, (F) > Cd(P). At t he ot her ext r eme, each
v~ is a r equest for a single page; t aki ng each
v~ = 1, we find ( af t er some al gebra) t ha t
w> wo = I + 2Ls / ( 2L + s - - 2)
is necessar y f or Ca(F) < Cd(P). For t he
nor mal l y he a vy dr um l oads ( L large) f ound
i n pagi ng syst ems, w0 _--~ s q- 1 is sl i ght l y
mor e t ha n a ful l dr um circumference.. I f we
r epeat t he anal ysi s t o i ncl ude t he cost of
swappi ng w pages out agai n at t i me qua nt um
end, we find w0 ~ 2s; for t ypi cal dr ums 2s
is appr oxi mat el y 8000 words, a subst ant i al
wor ki ng set. To sum up: as t he dr um l oad
var i es f r om t he f or mer ext r eme t o t he l at t er ,
t he syst em ent er s and exi t s st at es unf avor -
abl e t o swappi ng; even i n f avor abl e st at es,
swappi ng is cheaper onl y when worki ng
set s of subst ant i al size are moved. Our
anal ysi s does not account f or t wo ot her
f act or s: i t ma y be expensi ve t o find or mai n-
t ai n a suppl y of cont i guous sect ors i nt o
whi ch worki ng set s ma y be swapped, and
i t ma y be expensi ve t o i mpl ement bot h a
swappi ng pol i cy and a demand pagi ng
pol i cy in t he same syst em. Swappi ng t hus
appear s at best t o be of mar gi nal val ue i n a
demand pagi ng syst em. Thi s est abl i shes t he
t hi r d assert i on.
Now, l et Cp(P) denot e t he cost of pre-
pagi ng f r om dr um syst em P, and suppose
> 0 is t he pr obabi l i t y t ha t a pr epaged
page is not used. To pr epage f r om dr um P,
we woul d speci fy t he w pages as a gr oup
and add t he m t o t he dr um l oad L. I gnor i ng
to, t hi s cost s appr oxi mat el y wTp' , wher e
Tr ' is Tp eval uat ed at l oad L + w. Of t hese
w pages, ew were pr el oaded er r oneousl y, so
t her e will be ew addi t i onal page f aul t s;
assumi ng each of t hese repl aces an er r oneous
page wi t h a cor r ect one, t he cost for each
is wTe . Thus ,
Cp(P) = wTp' + ~w(wT~).
Af t er some al gebra, we find
w ~ w0 = (2L + s + 2) /
((1 -- 2e)(2L + s q- 2) -- 4)
is sufficient f or Cp(P) < Cd(P) t o hol d. Thi s
has t wo consequences. Fi r st , if is smal l and
L large, t hen w0 ~ 1, and pr epagi ng woul d
al most al ways be advant ageous. Second, in
or der t ha t t he denomi nat or of t he expressi on
f or w0 be posi t i ve, we r equi r e
e < (2L -b s -- 2) / ( 2L + s zr- 2).
Computing Surveys, Vol. 2, No. 3, September 1970
If e is not small and L is small, t hen w0 would
be large, and prepaging would not be ad-
vantageous. Since the foregoing argument
is very qualitative and based on average-
value arguments, we must be careful not to
at t ach too much significance to the particu-
lar expressions given. Our intention is
showing t hat the advantage of prepaging
may be very sensitive to the relations among
e, L, and s, and t hat careful analysis would
be required to assess its value in a given
system. (See [P1]. ) This establishes the
fourth assertion.
The foregoing discussion establishes also
t hat the performance of virtual memory
may depend strongly on the capacity of the
channel carrying the traffic of pages through
main memory. Although we have not studied
it, the reader should realize t hat several
parallel channels between main and auxiliary
memory (contrasted with the single channel
presumed above) would provide further
increases in capacity.
In general, the smaller the ratio of paging
traffic through memory to the system' s
capacity for handling it, the better the
performance of the virtual memory. To
minimize this ratio, we must (1) choose a
memory management policy to minimize the
rate at which a given program load generates
page faults, (2) modify program structure
to reduce the rate at which a given program
generates new page faults, and (3) provide
hardware support to increase the system' s
capacity for handling page traffic. These
three aspects are examined in detail in t he
following sections.
PROGRAM BEHAVIOR AND MEMORY
MANAGEMENT
Program behavior is among the least under-
stood aspects of computer system design
and analysis. And yet we need to model
program behavior if we are to have a sound
basis on which to predict a program' s future
memory needs or if we are to understand
how close resource allocation policies are to
being optimal.
Vir!ua! Memory 177
Replacement Algorithms
From now on we shall use N = {1, 2,
. . , n} to denote the pages of a given
program. A program' s dynamic behavior
may be described in machine independent
terms by its reference string
o -= r l r 2. . . r k . . . , rk E N, k > 1,
which is a sequence of those pages from N
which are referenced by the program (not
necessarily distinct). We suppose t hi s pro-
gram has been allocated a memory space of
size m, where 1 < m < n, and is to operate
in t hat space under paging. If t(rk) denotes
the time i nst ant at which page rk is refer-
enced, t hen the expected time E[t(r~+l) -
t(rk)] is h if rk is present in memory and
-~ T otherwise (see t he section on Basic
System Hardware). Therefore the expected
increment in space-time cost is
{
m~ if rk in
memory,
C(t(rk), t(rk+l)) = m(A q- T) otherwise.
When the page size is fixed and T > h
(typically, in fact, T >> ~), minimizing t he
total cost of running a program under
paging requires minimizing the number of
page faults. To understand what this en-
tails, we need a precise definition of replace-
ment algorithm.
A subset S of N such t hat S contains
m or fewer pages (written [S[ < m) is a
possible memory state, and 9lZ,, is the set
of all such S. A replacement algorithm
generally keeps records about the program' s
behavior; the st at us of its records will be
called a control State q, and Q is the set of
all such q. A replacement algorithm con-
figuration is a pair (S, q). If the configura-
tion is (S, q) and page i is referenced, a
new configuration (S/, qt) is entered. We
describe this behavior b y the allocation
mappi ng
g: ~Zm X Q X N- + ffg,~ X Q,
where
g(S, q, i) = (S' , q')
and i is in S' . Starting from an initial con-
figuration (So, q0), a replacement algorithm
Computing Surveys, Vol. 2, No. 3, September 1970
178 Pet er J . Denni ng
processes t he references rlr2 . . . rk by gen-
erating a sequence of configurations.
(So, qo), ( S, , q~), . . . , (S~, qk),
where
(Sk , qk) = g(Sk-1, qk-~ , r~), k > 1.
Thus a replacement al gori t hm A may be
described by specifying t he 3-tuple A =
(Q, q0, g).
Now if A is a demand paging replace-
ment algorithm, t hen whenever (S' , q' ) =
g(S, q, i ), t he memory st at e S p must satisfy
these properties:
If i E S then S ~ = S (no page fault).
If i ~ S and I S[ < m, then S' =
S [J {i} (page i added t o memory).
I f i ~ S a n d I SI = m, t h e n A selects
s omej E S a n d S' = (S - {j}) (J {i} (page
i replaces j ).
I t can be shown t hat , for any nondemand
paging algorithm A, one may construct a
demand paging algorithm A' t hat produces
no more faults t han A on every reference
string [A2, M2]. We are therefore justified
in restricting attention t o demand paging
algorithms. From now on, t he t erm "al -
gori t hm" specifically means "demand paging
replacement algorithm. "
Optimal Paging Algorithms
Suppose r~ . . - rk - . . rK is t he reference
string generated by a given run of a pro-
gram, and the reference moment t(rk) is
t hat of a page fault. If algorithm A requires
precise knowledge of the future (rk+l - - . r~)
t o make its replacement decision at t(rk),
A is an "unrealizable" algorithm. Otherwise,
if A bases its decision at t(rk) only on as-
sumptions about t he fut ure (e.g. probabil-
ities), A is a "realizable" algorithm. In
most practical applications, we must be
cont ent with realizable algorithms; un-
realizable ones would require "preprocess-
ing" the program and recording its refer-
ence string. Not only is this operation costly,
but t he record so obtained may well be
invalid, due to conditional branching.
As discussed in t he previous section, we
t ake as o u r opt i mal i t y criterion t he mini-
mization of the number of faults generated.
Since t he days of t he earliest paging machine,
people have reasoned t hat , t o minimize t he
number of faults, i t is necessary t o maxi-
mize t he times bet ween faults [K3]. There-
fore t he following has been t he accepted
PRINCIPLE OF OPTIMALITY. Let S =
{1 ~, 2 t, " " , m p} be the memory state at t i me
t, the moment of a page f aul t , and let t (i ~) > t
be the earliest moment at whi ch page i ' i s
next referenced. Defi ne v(i ' ) = t (i ' ) -- t.
Repl ace that page i ' f or whi ch ~.(i t) i s maxi -
mum. I f the f ut ure i s not precisely known,
replace t hat page i ' for whi ch the expected
t i me E[r(ir)] i s ma x i mum.
In t he case t hat we maximize E[~. (i ' )]--
t he case of realizable al gori t hms--we are
at t empt i ng only t o minimize t he expected
number of faults, rather t han t he actual
number of faults. Thus an optimal un-
realizable algorithm would produce fewer
faults t han an optimal realizable algorithm.
The principle of opt i mal i t y has great
intuitive appeal. Bel ady [B3] has used it t o
develop an optimal unrealizable algorithm.
Many ot her authors have developed various
optimal realizable algorithms, each depend-
ing on t he particular assumptions used t o
determine E[r(i' )]; for example, t he At l as
machine' s algorithm assumed most pro-
grams were looping and therefore generating
periodic reference strings [K3], and several
syst ems used an algorithm t hat supposes
E[r(i ' )] = t - t ' (i ' ) where t ' (i ' ) < t is t he
time i ' was most recently referenced (this
rule is called "least recently used"). We
shall not at t empt t o survey t he mul t i t ude
of paging algorithms t hat have been pro-
posed and studied, these being ampl y t reat ed
in t he literature [B3, B4, B6, C3, (38, D4,
D5, D9, H1, K5, J2, K3, 02, S2, S3, $5].
Despite its intuitive simplicity, t he Prin-
ciple of Optimality is known not to hold for
arbi t rary assumptions about reference string
structure and statistics. Even when it does
hold, proofs of this are difficult, and are
known only in simple cases [A2, M2].
Even though t he Principle of Opt i mal i t y
may not in fact be always optimal, it is a
good heuristic, and experience and experi-
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
ment al evidence i ndi cat e t ha t al gori t hms
based on t hi s principle give nearl y opt i mal
performance. Thi s evidence, suggested in
Fi gure 18, is abst r act ed from t he work of
Bel ady [B3], and of Coffman and Vari an
[C3]. Let F(A, m, oo) denot~ t he number of
faul t s generat ed as al gori t hm A processes
t he reference st ri ng ,o under demand pagi ng
in an i ni t i al l y empt y memor y of size m,
and define t he fault probability
f (A, m) = ~E~11 ~ Pr[oo](F(A, m, ~)/I oo I),
where Pr[~] denot es t he probabi l i t y of oc-
currence of ~, and I ~[ denot es t he l engt h
of oo. The curves f (A, m) for "r easonabl e"
al gori t hms A lie in t he shaded region of
Fi gure 18 (by "r easonabl e" we mean t ha t
t he assumpt i ons used t o det ermi ne E[r(i ' )]
in t he Principle of Opt i mal i t y are reason-
able). For compari son we have shown t he
rel at i ve position of f (A, m) for Bel ady' s
opt i mal unrealizable al gori t hm [B3]. The
poi nt is: for reasonable A, f (A, m) is much
more sensitive to m than to A. Therefore,
al t hough t he choice of pagi ng al gori t hm is
i mpor t ant , t he choice of memor y size is
critical.
Fi gure 18 brings out one ot her poi nt .
Occasionally in t he l i t erat ure one finds
anal yses of program behavi or based on t he
assumpt i on of randomness, i.e. t ha t each
page of a given program is equal l y likely t o
be referenced at any given reference. Thi s is
equi val ent t o t he assumpt i on t ha t E[r(i ' )] =
E[r ( j ' ) ] in t he Principle of Opt i mal i t y. I f
t hi s were so, t he f aul t probabi l i t y for ever y
realizable al gori t hm A would have t o be
f (A, m) = (n -- m)/n. Thi s si mpl y is not
t he case. Pr ogr ams t end t o reference cert ai n
pages heavi l y, ot hers l i ght l y, still ot hers
rarel y.
Cont r ar y t o i nt ui t i on, increasing t he
memor y size m may not al ways resul t in a
corresponding decrease in f (A, m); t ha t is,
f (A, m) mhy not be decreasing in m, as
suggested by Fi gure 18. The FI FO (first-in-
first-out) repl acement al gori t hm, for ' ex-
ample, is known t o exhibit an increasing
section in i t s f aul t pr obabi ht y curve, for
cert ai n reference strings [B6]. Mat t s on et
Virtual Memory 179
ffA,m )
!
"' , , / ~IZAaLE A
/ , , ~, , "~NOOM" a~oen.Ms
UNRIEALIZAEU[~ A ~ ~ ~'~xx x
0 1 I "" .......... ~: ~m
I n
Fro. 18. Fault probability
al. [M2] have discovered a ver y i nt erest i ng
class of repl acement al gori t hms, called
stack algorithms, whose f-curves are al ways
decreasing in m. These al gori t hms are de-
fined as follows. Let o be a reference string,
and let S(A, m, o) denot e t he memor y st at e
aft er A has processed ~ under demand pag-
ing in an i ni t i al l y empt y memor y of size m.
Al gori t hm A is a st ack al gori t hm if
S(A, m, oo) ~ S(A, m Jr 1, oo),
(i)
l <m<n ,
for every reference st ri ng w. Tha t is, t he
cont ent s of t he m-page memor y are al ways
cont ai ned in t he (m + 1)-page memor y, so
t ha t t he memor y st at es are "s t acked up"
on one anot her. The LRU (least-recently-
used) repl acement al gori t hm, for example,
is a st ack al gori t hm (to see t hi s, not e t ha t
S( LRU, m, ~) al ways cont ai ns t he m most
recent l y used pages). Consi der a st ack
al gori t hm A and a reference st ri ng ~x. I f
x is in S(A, m, w) - - t her e is no f aul t when
x is r ef er enced- - t hen by (i) x is also in
S(A, m Jr- 1, w); t hus increasing t he mem-
ory size can never resul t i n more page
faul t s, and f (A, m) mus t be decreasing i n
m for every st ack al gori t hm A. The class
of st ack al gori t hms cont ai ns all t he "reason-
abl e" algorithms, and t wo al gori t hms known
t o be opt i mal [A2, M2]. They are part i cu-
l arl y easy to anal yze [M2].
Computing Surveys, Vol. 2, No. 3, September 1970
180 Peter J. Denni ng
The Principle of Locality and the Working
Set Model
An i mpor t ant pr ogr am pr oper t y, al l uded
t o i n previ ous sections, is locality. Infor-
mal l y, l ocal i t y means t ha t duri ng any inter-
val of execution, a program favors a subset
of its pages, and t hi s set of favored pages
changes membershi p slowly. Local i t y is an
experi ment al l y observed phenomenon mani-
festing itself par t l y as a t endency for refer-
ences t o a given page t o cluster, par t l y in
t he shape of t he f ( A, m) curve in Fi gure 18
[B3, B4, D4, D5, D9], and par t l y in t he
r api di t y wi t h which a program acquires
cert ai n pages on demand at t he beginning
of a t i me qua nt um [C3, F2]. Local i t y is not
unexpect ed, by t he ver y nat ur e of t he way
programs are const ruct ed:
--Cont ext . At any given t i me a program
is operat i ng i n one of i t s modules, which
causes a concent r at i on of references in cer-
t ai n "r egi ons" or "l ocal i t i es" of address
space. For example', i t s i nst ruct i ons are
being fet ched from wi t hi n t he pages of some
subrout i ne, or its da t a are being f et ched
from t he cont ent of some specific dat a seg-
ment .
--Loopi ng. Pr ogr ams t end oft en t o loop
for a l ong t i me wi t hi n a small set of pages.
I n order t o render t he s t at ement of
l ocal i t y more precise, we i nt roduce t he
not i on of t he "reference dens i t y" for page i:
ai(k) = Pr[reference rk = i], i E N.
Thus 0 _< a~(k) _~ I and ~ at (k) = 1. Al-
t hough a program' s reference densities are
unknown (and perhaps unknowabl e), t he
defi ni t i on of "wor ki ng s et " gi ven below
obvi at es t he need for at t empt i ng t o measure
t hem. By a " r anki ng" of a program' s pages
we mean a per mut at i on R(k) = (1' , 2' ,
. . , n ' ) such t ha t a , ( k) ~ . . . ~_ a~,(k);
a r anki ng R(k) is " s t r i ct " if a, ( k) > . - . >
a~,(k). A "r anki ng change" occurs at refer-
ence k if R(/c - 1) ~ R(k); a "r anki ng life-
t i me" is t he number of references bet ween
r anki ng changes. Ranki ng lifetimes will
t end t o be long if t he a~(k) are slowly var y-
ing funct i ons of k.
PRINCIPLE OF LOCALITY. The ranki ngs
R(k) are strict and the expected ranki ng life-
times long.
Fr om t he principle of l ocal i t y comes t he
not i on of "worki ng set . " A pr ogr am' s work-
ing set at t he kt h reference is defined t o be
W(k, h) = {i E N I page i appears among
rk-h+l ". - rk}, h ~ 1.
I n ot her words, W(k, h) is t he " cont ent s "
of a " wi ndow" of size h looking backwards
at t he reference st ri ng from reference r k.
The worki ng set at t i me t is W(t , h) =
W(k, h) where t(r~) < t < t(rk+l). Page i is
expect ed t o be a member of t he worki ng set
if i t is referenced in t he window, i.e. if
k
ai (j) ~ 1.
j~k--h+l
(This equat i on, t oget her wi t h assumpt i ons
about t he at(k), could be used t o det er mi ne
a val ue for h. For example, if i t were as-
sumed t hat ai (k) = al and i t were decl ared
t ha t pages wi t h a~ < a0 for some given a0
ought not be expect ed as members of t he
worki ng set, t hen h = 1/ao .) Therefore, a
worki ng set is expect ed t o cont ai n t he "mos t
usef ul " pages; by t he principle of l ocal i t y i t
changes membershi p slowly.
Now suppose l ocal i t y holds and R(k) =
(1 p, 2' , . . . , n' ) . I f i ' is r anked hi gher t han
j ' (i.e. a~,(k) > aj, (k)) t hen E[r ( i ' ) ] <
E[r(j' )], and because r anki ng lifetimes are
long, t hi s rel at i on is expect ed not t o change.
Since i ' is more l i kel y t ha n j ' t o be in
W(k, h), t here follows:
WORKING SET PRINCIPLE. Suppose mem-
ory management operates according to the
following rule: A program may run i f and
only i f its worki ng set is i n memory, and a
page ma y not be removed i f it i s the member
of a worki ng set of a runni ng program. Then,
according to the principle of locality, this rule
is an implementation of the principle of opti-
mal i t y.
The worki ng set principle is more t ha n a
memor y management policy, for i t implies
a st rong correl at i on bet ween processor and
memor y allocation. I t s i mpl ement at i on does
not depend on measur ement of reference
Computing Surveys, Vol. 2, No. 3, September 1970
Virtual Memory 181
densities. This principle is used explicitly in
at least one computer system, the RCA
Spectra 70/46 [D2, O3, Wl].
Working sets exhibit a number of impor-
t ant properties. Let w(h) denote the expected
working set size, i.e. w(h) = E[ I W(t, h) I].
I t is shown in [D5] t hat , for h P_ 1,
(1) 1 _< w(h) < min {n, h},
(2) w(h) < w(h t - 1) (nondecreasing),
(3) w(h -q- 1) -q- w(h - 1) < 2w(h) (con-
cave down),
which give w(h) the general character of
Figure 19. The following is also shown in
[D5]. Let g(h) denote the probability t hat a
page, when referenced, is not in W(t, h).
Suppose h is increased by 1, so t hat a new
reference (rL-h) is included in the window;
the resulting change in the working set size
is
Zx W= { : otherwise.ifrt-hisntinW(t'h)'
But then E[5W] = g(h), and we have t he
i mport ant result t hat
~( h) = w( h + 1) - w( h) .
This suggests t hat measurements of a pro-
gram' s working set size function can be used
to obtain approximations to f(A, m), for
m = w(h) and working set strategy A. I t is
possible to relate w(h) to certain properties
of reference strings [D5], and t o use w(h) in
determining how much memory is required
in a given computer system [D7]. Finally,
let w(h, z) denote the expected working set
size (in pages) when the page size is z, and
apply the compression results of the section
on Compression Factor:
zlw(h, zl) _< z~w(h, z2) if zl < z2.
That is, a working set will comprise fewer
words for smaller page sizes.
The definition given above is not, of
course, the only possible definition for work-
ing set. As specified, the method for measur-
ing a working set is after the fact and its
reliability depends on the slowly varying
assumption about reference densities. The
method will fail to predict the imminent
w( h)
I
/ I
/
i i
~ h
O
FiG. 19. Expected working set si~e
presence in the working set of a page which
was not referenced in the window. This
definition is designed for systems where t he
future is unknown, where the principle of
locality holds most of the time, and where a
"maxi mum likelihood" estimate of the
future is sufficient. A still open question
concerns how to use "cont ext " and "loop-
i ng" properties, together with knowledge of
program structure, to predict before it is
referenced t hat a page will shortly become a
member of the working set.
Multiprogramming and Thrashing
Paging algorithms for multiprogrammed
memories normally lie at or between two
extremes:
1. Locally. The memory is partitioned
into "work spaces," one for each program.
The paging algorithm is applied independ-
ent l y in each work space. In particular, a
page fault in a given program can cause a
replacement only from its own work space.
The size of a work space remains fixed until
allowed to change by the system.
2. Globally. The paging algorithm is
applied t o the entire collection of running
programs, as if t hat collection were one
large program, without regard for which
pages belong to which programs. In particu-
lar, a page fault in a given program may
cause a replacement from arty program in
memory. The size of a program' s work space
is therefore randomly variable.
The working set principle, so formulated
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
182 Peter J. Denni ng
t ha t i t tells how memor y is t o be managed
under mul t i programmi ng, is a form of a
local policy. Indeed, global policies are in
general subopt i mal , par t l y because t here is
no way t o det ermi ne when memor y is "over-
cr owded, " and par t l y because t here is no
way t o guar ant ee t hat a program' s work
space is large enough t o cont ai n i t s worki ng
set even i f memor y is not "over cr owded. "
Mul t i pr ogr ammi ng under a global policy
is susceptible t o thrashing, a collapse of
performance t ha t ma y occur when memor y
(or par t s of memor y) is over commi t t ed
[D6]. Thr ashi ng is a compl i cat ed phenome-
non. At t he risk of oversimplification, we
shall derive a condi t i on t ha t est i mat es when
i t will occur. We assume (1) t ha t t he i t h
program in memor y has average work space
m~ and f aul t probabi l i t y f i (ml ) under t he
given global policy, where f~ is st at i onar y
over t he t i me i nt er val under consi derat i on;
and (2) for each i, f i ( m' ) ~ f~(m) whenever
t o t e m.
A " d u t y f act or " d(m) for a program occu-
pyi ng a work space of average size m ma y
be defined as follows: if f ( m) is t he pr ogr am' s
f aul t probabi l i t y, t hen t he expect ed number
of references bet ween faul t s is l / f (m); if each
memor y reference t akes expect ed t i me 5
(see t he section on Basic Syst em Har dwar e)
and each page t r anspor t t akes expect ed
t i me T, t hen t he expect ed fract i on of t i me
t hi s pr ogr am spends in execut i on is
d(m) [5/ f ( m) l / [5/ f ( m) + T],
o~ = T/ ~.
= 1 / [ 1 --~ af ( m) ] ,
Using condi t i on (2) above, i t is not difficult
t o show t hat , if m' < m,
0 ~ d(m) -- d(m' ) < a( f ( m' ) -- f ( m) ) . (i)
I f d(m) -- d(m' ) is near i t s upper bound and
a is large, a rel at i vel y small change in work
space size will be reflected as a large change
in d. Thi s is necessary t o i nduce t hrashi ng.
Now i magi ne t he following condi t i ons
hol di ng for an M-page mul t i pr ogr ammed
memor y using a global policy. I ni t i al l y t here
are k - 1 programs in memor y, t he i t h pro-
gram occupies a work space of average size
mi ~ 1, and ml -b " - + mk-1 = M. When
t he kt h program is i nt roduced, i t is gr ant ed
mk' pages and t he global policy changes t he
remai ni ng mi t o mi ' < m~. Let t i ng Dj de-
not e t he t ot al expect ed processing efficiency
when j programs are in memor y, we .have
k--1
Dk-, = ~ d, (mi ),
k
Dk = ~ di(m~' ).
i =l
Thr ashi ng occurs if Dk << D~_1,1 i.e. t he
addi t i on of one more pr ogr am triggers a
collapse of processing efficiency. Using (i)
we find
k--1
Dk-j -- Dk < a ~ (f~(m~') -- f~(m~))
i ~l
-- dk(mk' ) (it)
aFo -- dk(mk' ).
Now if t he quant i t y Dk_~ -- Dk is near i t s
upper bound and aF0 is not small, t hen i t is
possible t o obt ai n Dk_l << Dk. Exper i ment s
on t he RCA Spect ra 70/46 comput er sys-
t em, for which a > 104 (a dr um auxi l i ary
memor y) , show t ha t t hi s condi t i on is easy
t o i nduce [D2]. Conversel y, we can pr event
t hr ashi ng if we can guar ant ee t ha t aF9 is
small, which may be done by using fast er
auxi l i ary memor y or by operat i ng programs
wi t h space allocations which var y onl y in
ranges where F0 is small.
Now suppose a worki ng set policy is in
effect. Let t he r andom variable ~(h~) denot e
t he worki ng set size of pr ogr am i for wi ndow
size h~, and let g~(h~) denot e t he probabi l i t y
t ha t a page is not in t he working set. Be-
cause t he pages wi t h hi ghest reference den-
sities are most likely t o be members of t he
worki ng set, gi is decreasing, i.e. gi(hi) >
g~(h~ + 1). The dut y fact or d~(h~) for pro-
gram i under a worki ng set policy satisfies
d,(h~) > 1/[1 + age(hi)I,
where t he i nequal i t y hol ds because a page
not in t he worki ng set ma y still be in t he
memor y, so t ha t g~(h~) is at least as large
as t h e f aul t probabi l i t y. Since g~ is decreas-
Notation x << y means "x is much less than y."
Comput i ng Surveys, Vol. 2, No, 3, Sept ember 1970
ing, we may always choose h~ large enough
so t hat g~(hO _< go for some given go, 0 <
go <_ 1 ; therefore we may guarantee t hat
do ~ 1/(1 + ago) _~ ddh~) _< 1.
In other words, we may always choose h~
large enough t hat program i operates at or
above the desired level do of efficiency.
(Normally, we would choose do so t hat the
relation do ~ 1 is false.) This implies t hat
l~do < Dk _< k. (iii)
If we are considering adding t he kth pro-
gram to memory, we may do so if and only if
k--1
w,(hk) _< M - ~ ~i(h~),
i --1
i.e. there is space in memory for its working
set. Assuming t hat do << 1 is false, the addi-
tion of the kth program cannot cause thrash-
ing. Suppose it does, i.e. suppose Dk <<
Dk_~ ; by (iii) we have
kdo < Dk << Dk-1 < lc,
which yields the contradiction do << 1.
Thus working set policies may be used to
prevent thrashing. Experiments on the RCA
Spectra 70/46 computer system appear to
verify this [D2].
PROGRAM STRUCTURE
Careful at t ent i on to algorithm organization
and program structure can improve the
performance of virtual memory systems.
There are two ways in which this can be
accomplished: distributing program code
properly into pages, and improving pro-
gramming style.
Program code is normally assigned to
pages simply by assigning the first z words
to page 1, the next z words to page 2, and
so on. There is considerable evidence t hat
this may be far from satisfactory. Comeau
[C7] describes an experiment in which a
program consisting of many subroutines
was paged, first with t he subroutines in
alphabetic order, then with the subroutines
grouped together according as t hey were
Virtual Memory 183
likely to call one another; there was a re-
markable reduction in the number of page
faults using t he l at t er method. McKellar
and Coffman [M4] have studied how matrix
elements should be assigned to pages and
how st andard matrix operations could be
organized to give better performance under
paging; t hey too report a rather remarkable
improvement in certain cases.
Informally, the code distribution problem
is: How can the compiler (or t he subroutine
linker) be employed t o distribute program
code and dat a into pages in order to improve
locality and obtain small, stable working
sets? Formally, the code distribution prob-
lem may be stated in the following way.
A program is regarded as a directed graph
G whose nodes represent instructions or
dat a and whose edges represent possible
single-step control transfers. Wi t h edge
(i, j ) is associated a cost c~i >_ 0 of travers-
ing t hat edge (c~ might, for example, repre-
sent the probability t hat (i, j ) will be used).
Given a page size z ~ 1, a pagination of the
program is a partition of the nodes of G
into disjoint sets X~, . . . , Xr such t hat Xk
contains at most z nodes, 1 < k < r. Each
X~ will be placed on its own page. For a
given pair of pages (X, X/), let
v(x, x' )= 52 52 c,j
i EX j EX ~
denote the t ot al cost of MI edges passing
between X and X/. The cost of the pagina-
tion X1, , X~ is t hen
C(X,, . . . ,Xr) = ~ V(X,,Xj ).
A pagination is optimal if it achieves mini-
mal cost. Calculating an optimal pagination
for a given program is in general a hope-
lessly complex computation, and relatively
simple algorithms are known only in special
cases [K2, R1]. Even then, t he prospective
user of such a scheme would be faced with
the problem of deciding whether he would
be executing the optimized code sufficiently
often t hat the long-term savings would
balance the initial high cost of obtaining
the optimized code.
One must be careful with this sort of
approach. However attractive the mathe-
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
184 Peter J. Denning
matics involved, the results may not be
particularly useful except in certain obvious
cases such as those mentioned above. If t he
t rend t oward increased use of modular
programming continues, t he value of using
a compiler t o determine an optimal pagina-
tion is questionable: (1) program modules
t end t o be small, and very often fit on their
own pages; and (2) in contradiction to t he
assumption t hat the code optimizer must
know the connectivity structure of t he
entire program, the compiler of a module
may not know the internal structure of any
ot her module. (If it did, the ver y purpose of
modular programming would be defeated.)
The optimization process cannot, therefore,
be invoked prior t o loading time; and if t he
t rend t oward dat a dependent program
structures continues, there is some question
whether even the loader can perform mean-
ingful optimization.
Improving programming style t o improve
locality is an almost intangible objective
and is something about which little is known
or can be said [K6]. A few experiments
show t hat locality (and therefore paging
behavior) is strongly a function of a pro-
grammer' s style, and it is possible to im-
prove many programs significantly by rela-
tively minor alterations in strategy, altera-
tions based on only a slight knowledge of
the paging environment [B9, $2]. I t is not
known, however, whether programmers can
be properly educat ed and inculcated with
t he "ri ght " rules of t humb so t hat t hey
habi t ual l y produce programs with "good"
locality. If any such education is t o be
fruitful for a large class of programmers, it
PROCESSOR
. . . . . . . S L kVE CON N ECT I ON S
- - - D I ST RI B UT I VE CON N ECT I ON S
Fro. 20. Memory hierarchy structure
must teach techniques t hat may be applied
wi t hout knowledge of machine details (page
size, memory size, and t he like). Highly
st ruct ured programming languages, where
the "cont ext " (see t he section on The Prin-
ciple of Locality and t he Working Set
Model) is readily detectable at the machine
level, may be t he answer; in other words,
the programming language would "force"
t he programmer into the "correct " style.
The programming language ALGOL, which
makes heavy use of a st ack during execu-
tion, is an example of this; t he working set
will surely contain the information near t he
t op of the stack, and is therefore easily
measured. Much more sophisticated ap-
proaches have been conceived [D14].
HARDWARE SUPPORT
We have seen t hat the three principal po-
tential difficulties with multiprogrammed,
paged memory systems are fragmentation,
thrashing, and t he high space-time cost of
loading working sets into memory under
demand paging. These three problems are
partially at t ri but abl e to the large speed
ratio T/A bet ween the main and auxiliary
memory; if this ratio is large, it forces large
page sizes in order to make page t ransport
operations efficient, it makes processing
efficiency ver y sensitive to fluctuations in
fault probability, and it causes t he space-
time cost of a single page-transport opera-
tion to be very high. Therefore, one aspect
of improving hardware for virtual memory
concerns t he reduction of this ratio.
The literature reports two directions in
which approaches to reducing the ratio
T/A have proceeded, to which we shall
refer as slave memory ("cache" memory)
[F5, L2, W3, W4] and distributive memory
[A3, D8, F1, L1, V1]. Bot h approaches
employ a memory hierarchy (Figure 20)
consisting of k "levels"; levels M1, . . .
Mk_l are electronically accessed (e.g. core
memory, thin film memory, or silicon-
register memory), and level Mk is mechan-
ically accessed (e.g. drum or disk). The
electronic levels may be accessed wi t hout
Computing Surveys, Vol. 2, No. 3, September 1970
latency time. Generally, the lower the num-
ber of the level, the faster its speed, the
higher its cost, and the lower its capacity.
The distinguishing feature is t hat slave
memory permits processing only from level
M1, whereas distributive memory allows
processing from any of the electronic levels
M1, - . . , Mk-1.
Typically, the combined capacity of t he
electronic levels in these approaches is large
enough to hold all the information of all
active programs. Therefore, the transport
time for a page among the electronic levels
is small, because the speed ratios between
adjacent levels can be made small. Accord-
ingly, a hierarchical memory organization
of this kind can achieve the objectives re-
quired to make paged virtual memory per-
form well.
The slave memory approach [W3] was
first implemented as the "cache store" on
the IBM 360/85 [L2]. This approach is so
named because information transfers among
levels are entirely controlled by activity in
the ("master") level M~ . The rules of opera-
tion are:
1. Whenever a page is stored in Mi , there
is a copy of it in each of M~+i, , Mk-1.
Whenever a page in M1 is modified, all
copies of it in the lower levels must be
modified likewise.
2. Whenever a page not in Mj is refer-
enced, a request for it is sent to the lower
levels; the retrieval time depends on the
"di st ance" to the "nearest " level containing
a copy of the required page.
3. Whenever M~ is full and a new page is
brought in from M~+j, a replacement policy,
usually least recently used, is invoked to
select a page to be deleted (since there is
already a copy in M~+i, there is no need to
move the displaced page).
The principal advantage of this organiza-
tion is t hat a program' s working set will
rapidly accumulate in M~ and be retained
there; accesses will thus be completed at
nearly the speed of Mj . A second advantage
is t hat , because transport times are small,
pages may be small, and all the advantages
of small pages are accrued. A t hi rd ad-
Virtual Memory . 185
vantage is t hat the mechanism is simple
enough to be implemented almost entirely in
hardware [W3]. A fourt h advantage is t he
possibility of implementing certain associa-
tive processing operations in the main
level [$6].
Many modern processors employ an
"instruction st ack, " which is a small num-
ber of registers (usually no more t han 32)
t hat store the most recently referenced in-
structions of a program. Not only does this
stack permit "lookahead, " it' acts as a small
slave memory t hat allows processing to
proceed at nearly register speed for loops
t hat are contained in the stack [W3]. The
most notable examples of slave memory
implemented as discussed above are the
cache memory [L2] on the I BM 360/85,
I BM 360/195, and CDC 7600. These sys-
tems use k = 3, Mj being a silicon-register
memory with cycle time about 0.1 gsec and
Ms a core memory with cycle time about
1 gsec. The level Mj is about 32K bytes
capacity, and has been found substantial
enough to accumulate the working sets of
all but the largest programs. Even if t he
working set cannot be contained in M1,
performance is not appreciably degraded
because the speed ratio between M1 and
M2 is small.
In the distributive memory approach, the
processor may access information stored in
any of the electronic levels. Thus the pages
of a given program may be distributed
among the various levels while being proc-
essed. Generally, the more frequently a
page is referenced, the higher should be the
level in which it is stored. The most notable
example of such a system is t hat at Carnegie-
Mellon University IF1, L1, V1], which uses
k = 3; M~ is a st andard core memory with
cycle time about 1 ~sec and Ms a large
capacity store (LCS) with cycle time about
8 ~sec.
The distributive memory system presents
certain sticky implementation problems not
found in the slave memory system. The
worst is a requirement t hat there be a policy
to determine when a page should be moved
to a higher (or lower) level. These policies
are generally based on a tradeoff between
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
186 Peter J. Denning
the cost of not moving t he page and running
at slower speed, and t he cost of moving t he
page; t hey generally require some estimate of
each page' s reference densi t y for these de-
cisions, t he estimates being obt ai ned by
preprocessing [C1], by measurements t aken
in a previous time quant um [F1], or dy-
namically [D8]. Syst ems using dynamic
measurement techniques require additional
mechanism t o avoid instability [D8].
Which of t he t wo approaches--sl ave or
distributive memor y- - i s superior is an un-
settled question. That t he implementation
problems of distributive memory seem more
severe leads one to suspect t hat perhaps t he
slave memory approach may be t he bet t er
way t o use the hardware.
Reducing the ratio T/ A is not alone
sufficient to improve performance of vi rt ual
memory systems. A second aspect of im-
proving hardware for these syst ems con-
cerns mechanisms for obtaining measure-
ment s useful in memory allocation. Most
syst ems implement page t abl e entries with
one or more of these ext ra bits present:
1. Modified bit. Set t o 1 if and only if
t he page was modified since being placed in
memory. If this bi t is 0, t he page may be
deleted rather t han replaced, assuming there
is a copy in a lower level of memory.
2. Use bit. Set to 1 whenever t he page is
referenced, and t o 0 by a usage metering
routine. The metering routine can compile
statistics on page use by reading these bits.
3. Unused bit. Set to 1 when a page is
placed in memory and to 0 the first time it
is referenced. This hit signifies t hat t he page
has not yet been referenced by t he program
t hat demanded it, and should not be re-
moved from memory at least until t hat time.
The use bits may serve to determine a
working set or t o calculate reference den-
sities. Counters can also be used for this
purpose [D8]. If the addressing mechanism
contains a large enough associative memory
t hat its contents remain stable, then t he
pages entered there may be regarded as t he
program' s working set; similarly, t he pages
which accumulate in t he level M1 of t he
slave memory may be regarded as t he pro-
gram' s working set.
A third aspect of improving vi rt ual mem-
ory hardware concerns t he nat ure of t he
addressing mechanisms. Difficulties have
occurred in virtual memories where informa-
tion is potentially sharable among distinct
address spaces [B7, D13]. Here each seg-
ment may have two names: a "local" name
which serves to identify it within a given
address space, and a "gl obal " name which
serves t o identify it systemwide. Local
names are interpreted in t he usual way by
hardware (see the section on Implementa-
tion of Virtual Memory), and global names
are i nt erpret ed by software (e.g. "file direc-
tories"). The mechanism for converting
global names to local names is quite involved
and time consuming [B7, D1]. The solution
appears t o require t hat every segment have
one, system-wide name which may be in-
t erpret ed by hardware at every level of
memory [D141.
CONCLUSIONS
We began this survey of virtual memory
syst em principles by tracing the; hi st ory
and evolution of the forces t hat compelled
dynamic storage allocation, i.e. desires for
program modularity, machine independence,
dynamic dat a structures, eliminating manual
overlays, multiprogramming, and time-
shari n~-Amoag the mo t ~ e g _ ~ t solutions
to t he d ~ n a mi c ~ f ~ g ~ allocation~problem is
vi rt ual memory, wherein a-_or.p_gr:A_~._2,r--Js--
.given the illusion t hat his address spa c, e_is
.~.e"- ihem(J~ space. There ~-aar6 t wo basic
ap~r oa~s ~o- i ~p- I ~ment i ng t he automatic
translation of addresses from address t o
memory space, these being ~ment at i ox~5
and .paging ) since segmentation ~ i - r e O
by programmers and paging b y system
implementers, t he best implementation c()n~- "
bines t he two. We compared "pur ~! seg=_.
ment at i on with paging, and ffoa!ng~p_gy~
memory syst ems generally ,.J.~pexic[r except
for three potential difficulties-'~ (1) suscepti-
bility t o low storage utilization for large
page sizes, (2) propensity t oward thrashing
under multiprogramming, and (3) t he high
cost of loading working sets under demand
paging at t he st art of a time quant um. One
Comput i ng Surveys, Vol. 2, No. 3, Sept ember 1970
probl em wi t h all i mpl ement at i ons of vi rt uM
memor y in which t he address space is much
l arger t han t he memor y space is pot ent i al
misuse by pr ogr ammer s clinging undul y t o
t he i dea t hat space and t i me ma y be t raded.
Thi s l ast st at ement must , however, be in-
t er pr et ed carefully. Pr ogr ammer s who have
been war ned t ha t t he space-t i me t radeoff
does not hold, and have gone t o t he ext r a
work of reduci ng t he t ot al amount of ad-
dress space empl oyed, have oft en increased
t he size of t he worki ng set. The obj ect i ve is
t o have a small, stable, slowly changi ng
worki ng set. I f t hi s is achieved, t he amount
of address space empl oyed is i mmat eri al .
These probl ems can be controlled, but re-
quire har dwar e suppor t above and beyond
t hat offered by many cur r ent systems. Since
a memor y syst em is more t han mere imple-
ment at i on of an address map, we i ncl uded u
s t udy of t he principles of opt i mal replace-
ment policies, and f ound t ha t t he ,worl~ing
..get prJ.aai-phb t oget her with t he .prk:w,l.ple--~f
l o c a ~ is an i mpl ement at i on of t he Pri n-
ciple of Opt i mal i t y. By st at i ng a met hod
wher eby one ma y det ermi ne each pr ogr am' s
worki ng set, t hi s principle implies t ha t one
ma y t ake steps t o avoi d over commi t ment
of memor y, and t hrashi ng.
ACKNOWLEDGMENTS
I am deepl y grat eful t o Jack B. Denni s
(of MI T) , t o Ber nar d A. Galler and Bruce
W. Ar den (bot h of t he Uni ver si t y of Michi-
gan), and t o Davi d Sayr e (of I BM T. J.
Wat s on Research Cent er), whose penet r at -
ing comment s pr oved i nval uabl e in i mprov-
ing t he manuscri pt . I shoul d also like t o
t hank John E. Pomer anz (of t he Uni ver si t y
of Chi cago) for suggest i ng some refinements
in t he section on Demand Pagi ng.
REFERENCES
A1. ABATE, J., AND ].)UBNER, It. Optimizing the
performance of a drum-like storage. IEEE
Trans. C-18, 11 (Nov. 1969), 992-997.
A2. AHO, A. V., DENNING, P. J., ANn ULLMAN,
J. 1). Principles of optimal page replace-
ment. Computer Science Tech. Rep. No. 82,
Princeton U., Princeton, N. J., Jan. 1970.
Vi r t ual Me mor y 187
A3. ANACKER, W., AND WANG, C. P. Per f or mance
evaluation of computing systems with mem-
ory hierarchies. IEEE Trans. EC-16 (Dec.
1967), 764-772.
A4. ARDEN, B. W., AND BOETTNEi% D. Measure-
ment and performance of a multiprogram-
ruing system. Proc. Second ACM Symp. on
Operating Systems Principles~ Princeton, N. J.,
Oct. 20-22, 1969, pp. 130-146.
A5. , GALLER, B. A., O'BmEN, T. C., AND
WESTERVELT, F. I~I. Program and address-
ing structure in a time-sharing environment.
J. ACM 18, 1 (Jan. 196fi), 1-16.
B1. BASKETT, F. , BROWNE, J. C. , AND RAIKE,
W. M. The management of a multi-level
non-paged memory system. ProP. AFIPS
1970 Spring Joint Comput. Conf., Vol. 36,
pp. 459-465.
B2. BATSON, A., Ju, S., ANn WOOD, D. Measure-
ments of segment size. Proc. Second ACM
Syrup. on Operating Systems Principles,
Princeton, N. J., Oct. 20-22, 1969, pp. 25-29.
Also, Comm. ACM 13, 3 (March 1970), 155-159.
B3. BEL~t)Y, L. A. A study of replacement
algorithms for virtual storage computers.
IBM Syst. J. 5, 2 (1966), 78-101.
B4. . Biased replacement algorithms for
multiprogramming. Rep. NC697, IBM T. J.
Watson Res. Center, Yorktown Heights,
N. Y., March 1967.
B5. - - - - ANn KUEHNER, C. J. Dynamic space
sharing in computer systems. Comm. ACM
12, 5 (May 1969), 282-288.
B6. - - , NELSON, R. A., AND SHEDLER, G. S.
An anomaly in the space-time characteristics
of certain programs running in paging ma-
chines. Comm. ACM 12, 6 (June 1969), 349-
353.
B7. BENSOUSSAN, A. , CLINGEN, C. T., ANn
DAImY, R. C. The Multics virtual memory.
Proc. Second ACM Syrup. on Operating
Systems Principles, Princeton, N. J., Oct. 20-
22, 1969, pp. 30-42.
B8. Bol~aow, I). G. , AND MURPHY, D. L. Struc-
ture of a LISP system using two-level stor-
age. Comm. ACM 10, 3 (March 1967), 155-159.
B9. BRAWN, B., AND GUSTAVSON, F. Program
behavior in a paging environment. Proc.
AFIPS 1968 Fall Joint Comput. Conf., Vol.
33, pp. 1019-1032.
B10. BURRO~'~HS CORPOR~TtON. The descriptor
--A definition of the B5000 information
processing system. Burroughs Corp., 1961.
C1. CHEN, Y. C. Selective transfer analysis.
Rep. RC-1926, IBM T. J. Watson Research
Center, Yorktown Heights, N. Y., 1968.
C2. COFFMAN, E. G. , JR. Analysis of a drum in-
put/output queue under scheduled operation
in a paged computer system, d. ACM 16, t
(Jan. 1969), 73-90.
c a . - - - - , AND VARIAN, L. C . Further experimen-
tal data on the behavior of programs in a pag-
ing environmen6. Comm. ACM 11, 7 (July
1968), 471-474.
C4. COHEN, J. A. Use of fast and slow memories
in list processing languages. Comm. ACM 10,
2 (Feb. 1967), 82-86.
Computing Surveys, Vol. 2, No. 3, Sep~mber 19770
188 Pet er J . De n n i n g
C5. COHEN, L. J. Stochastic eval uat i on of a
st at i c storage allocation. Comm. ACM ~4, 10
(Oct. 1961), 460-464.
C6. COLLINS, G. O., JR. Experience i n automatic
storage allocation. Comm. ACM 4, 10 (Oct.
1961), 436-440.
C7. COMEAU, L. A st udy of the effect of user
program opt i mi zat i on i n a paging system.
ACM Symp. on Operating System Principles,
Gat l i nburg, Tenn. , Oct. 1-4, 1967 (7 pp.).
C8. CORBAT6, F. J. A paging experiment wi t h
the Multics system. Rep. MAC-M-384, MI T
Proj ect MAC, Cambridge, Mass., May 1968.
D1. DALEY, R. , AND DENNIS, J. B. Vi r t ual
memory, processes, and sharing i n multics.
Comm. ACM 11, 5 (May 1968), 306-312.
D2. DEMEIs, W. M. , AND WEIZER, N. Meas ur e-
me nt and analysis of a demand paging time
shari ng system. Prec. 24th Nat . Conf.
ACM, ACM Pub. P-69, 1969, pp. 201-216.
D3. DENNING, P. J. Effects of scheduling on
file memory operations. Prec. AFI PS 1967
Spring Joi nt Comput. Conf., Vol. 30, pp.
9-21.
D4. . The working set model for program
behavior. Comm. ACM 11, 5 (May 1968),
323-333.
D5. . Resource allocation in multiprocess
computer systems. Tech. Rep. MAC-TR-50,
MI T Project MAC, Cambridge, Mass., 1968
(Ph. D. thesis).
Thr as hi ng: I t s causes and prevent i on.
D6. Proc'. AFI PS 1968 Fal l Joi nt Comput. Conf.,
Vol. 33, pp. 915-922.
Equi pment configuration in balanced
DT. computer systems. I EEE Trans. C-18
(Nov. 1969), 1008-1012.
D8. AND BRUNO,J. L. On the management
of multilevel memories. Computer ~cience
Tech. Rep. 76, Pri ncet on U., Pri ncet on,
N. J., April 1969.
D9. - - , CHEN, Y. C., AND SHEDLER, G. S. A
model for program behavior under demand
paging. Rep. RC-2301, IBM T. J. Watson
Res. Center, Yorktown Heights, N. Y.,
Sept. 1968.
D10. DENNIS, J. B. Program st ruct ure i n a
multi-access computer. Tech. Rep. MAC-
TR-11, MI T Project MAC, Cambridge,
Mass.
Dl l . . Segmentation and the design of mul-
tiprogrammed computer systems. J. ACM
12, 4 (Oct. 1965), 589-602.
D12. - - AND GLASER, E. L. The st ruct ure of
on-line i nformat i on processing systems.
Prec. Second Congress on I nf or mat i on Syst.
Sci., Spart an Books, Washington, D. C.,
1965, pp. 5-14.
D13. - - ~ND VAN HORN, E. C. Programmi ng
semantics for multiprogrammed computa-
tions. Comm. ACM 9, 3 (March 1966), 143-
155.
D14. - - . Programmi ng generality, parallelism
and computer architecture. Prec. I FI P
Congr. 1968, Vol. 1, Nort h-Hol l and, Amster-
dam, 1969, pp. 484-492 (Comput at i on Struc-
tures Group Memo 32, MI T Project MAC,
Cambridge, Mass., Aug. 1968).
F1. FIKES, R. E., LtUER, H. C., AND VAREHA,
A. L., JR. Steps toward a general-purpose
time-sharing system using large capacity
core storage and TSS/360. Prec. 23rd Nat.
Conf. ACM, ACM Pub. P-68, 1968, pp. 7-18.
F2. FINE, G. H. , JACKSON, C. W. , AND MclSAAC,
P. V. Dynami c program behavior under
paging. Prec. 21st Nat . Conf. ACM, ACM
Pub. P-66, 1966, pp. 223-228.
F3. FOTHERINGHAM, J. Dynami c storage allo-
cation in the Atlas computer, including an
automatic use of a backing store. Comm.
ACM 4, 10 (Oct. 1961), 435-436.
F4. FREIBERGS, I. F. The dynamic behavior of
programs. Prec. AFI PS 1968 Fal l Joi nt Com-
put . Conf., Vol. 33, pp. 1163-1168.
F5. FUCHEL, K., AND HELLER, S. Considerations
i n the design of a mul t i pl e computer system
wi t h extended core storage. Comm. ACM 11, 5
(May 1968), 334-340.
H1. HELLERMAN, H. Complementary replace-
ment - - A met a scheduling principle. Prec.
Second ACM Syrup. on Operating Systems
Principles, Princeton, N. J., Oct. '20-22, 1969,
pp. 43-46.
H2. HOLT, A. W. Program organization and
record keeping for dynami c storage alloca-
tion. Comm. ACM 4, 10 (Oct. 1961), 422-431.
I1. ILIFFE, J. K. Basic Machine Principles.
American Elsevier, New York, 1968.
I2. - - AND JODEIT, Z. G. A dynamic storage
allocation scheme. Comput. J. 5 (Oct. 1962),
200-209.
J1. JOHNSTON, J. B. The st ruct ure of multiple
act i vi t y algorithms. Proc. Thi rd Annual
Pri ncet on Conf., Pri ncet on, N. J., March
1969.
J2. JONES, R. M. Fact ors affecting the efficiency
of a vi rt ual memory. I EEE Trans. C-18, 11
(Nov. 1969), 1004-1008.
K1. KELLEY, J. E., JR. Techniques :for storage
allocation algorithms. Comm. ACM .4, 10
(Oct. 1961), 449-454.
K2. KERNIGtIAN, B. W. Opt i mal segment at i on
points for programs. Prec. Second ACM
Symp. on Operating Systems Principles,
Princeton, N. J., Oct. 20-22, 1969, pp. 47-53.
K3. KILBURN, T. , EDWARDS, D. B. G., LANIGAN,
M. J. , AND SUMNER, F . H. One- l evel s t or age
system. I RE Trans. EC-11, 2 (April 1962),
223-235.
K4. KNUTII, D. E. The Arl of Computer Program-
ming, Vol. I. Addison-Wesley, Reading,
Mass., 1968, pp. 435-455.
K5. KUCK, D. J. , AND LAWRIE, D. H. The use
and performance of memory hierarchies: A
survey. Tech. Rep. No. 363, Dep. of Comput er
Sci., U. of Illinois, Urbana, Ill. , Dec. 1969.
K6. KUEHNER, C. J. , AND RANDELI,, B. Demand
paging in perspective. Prec. AFI PS 1968 Fall
Joi nt Comput. Conf., Vol. 33, pp. 1011-1018.
L1. LAUER, H. Bulk core in a 360/67 time shari ng
system. Prec. AFI PS 1967 Fall Joi nt Comput.
Conf., Vol. 31, pp. 601-609.
Computing Surveys, Vol. 2, No. 3, September 1970
L2. LIPTAY, J. S. The cache. IBM Syst. J. 7, 1
(1968), 15-21.
M1. MACKENZIE, F. B. Automated secondary
storage management. Datamation 11, 11
(1965), 24-28.
M2. MATTSON, R. L., GECSEI, J., SLUTZ, D. R.,
AND TRAIGER, I . W. Eval uat i on Techniques
for Storage Hierarchies. IBM SysL J. 9, 2
(1970), 78-117.
M3. MCCARTHY, J., COR~AT6, F. J., ANn DAG-
GETT, M. M. The Li nki ng Segment Sub-
rogram Language and Li nki ng Loader.
omm. ACM 6, 7 (July 1963) 391-395.
M4. McKELLAR, A., AND COFFMAN, E. G. The
organization of matrices and matrix opera-
tions i n a paged mul t i programmi ng environ-
ment. Comm. ACM I2, 3 (March 1969),
153-165.
M5. MI T. Report of the long range computa-
tion st udy group, April 1961.
O1. O'NEILL, R. W. A preplanned approach to a
storage allocating computer. Comm. ACM .~,
10 (Oct. 1961), 417.
02. . Experience using a time sharing mul t i -
programming system with dynamic address
relocation hardware. Proc. AFI PS 1967 Spring
Joi nt Comput. Conf., Vol. 30, pp. 611-621.
03. OPPENI~EIMER, G., AND WEIZER, N. Re-
source management for a medium scale time
sharing operating system. Comm. ACM i i , 5
(May 1968), 313-322.
P1. PINKERTON, T. Program behavior and con-
trol in vi rt ual storage computer systems.
CONCOMP Proj ect Rep. No. 4, U. of Mich.,
April 1968 (Ph.D. thesis).
P2. POOLE, P. C., AND WHITE, W. Machine-
i ndependent software. Proc. Second ACM
Symposium on Operating Systems Principles,
Princeton, N. J., Oct. 20-22, 1969, pp. 19-24.
R1. RAMAMOORTHY, C. V. The analytic design of
a dynamic look ahead and program segment-
ing system for multiprogrammed computers.
Proc. 21st Nat. Conf. ACM, ACM Pub. P-66,
1966, pp. 229-239.
R2. RANDELL, n. A note on storage fragmenta-
t i on and program segmentation. Comm.
ACM 1~, 7 (July 1969), 365-369.
R3. AND KEHNER, C. J. Dynamic storage
allocation systems. Comm. ACM 11 (May
1968), 297-305.
Vi rt ual Me mor y 189
R4. RISKIN, B. N. Core allocation based on
probability. Comm. ACM 4, 10 (Oct. 1961),
454-459.
$1. SAMS, B. H. The case for dynamic storage
allocation. Comm. ACM 4, 10 (Oct. 1961),
417-418.
$2. Shi nE, D. Is automatic folding of programs
efficient enough to displace manual ? Comm.
ACM 12, 12 (Dec. 1969), 656--660.
83. SHEMER, J. E., AND GUPTA, S. C. On t he
design of Bayesian storage allocation algo-
ri t hms for paging and segmentation. IEEE
Trans. C-18, 7 (July 1969), 644-651.
$4. - - AND SHIPPEY, B. St at i st i cal analysis of
paged and segmented computer systems.
IEEE Trans. EC-15, 6 (Dec. 1966), 855-863.
$5. SMITH, J. L. Mul t i programmi ng under a
page on demand strategy. Comm. ACM 10,
10 (Oct. 1967), 636--646.
$6. STONE, H. S. A logic-in-memory computer.
IEEE Trans. C-19, 1 (Jan. 1970), 73-78.
Vl. VAREHA, A. L., RUTLEDGE, n . M., AND GOLD,
M. M. Strategies for st r uct ur i ng two-level
memories in a paging envi ronment . Proc.
Second ACM Syrup. on Operating Systems
Principles, Princeton, N. J., Oct. 20-22, 1969,
pp. 54-59.
Wl. WEINGARTEN, A. The Esehenbach drum
scheme. Comm. ACM 9, 7 (July 1966), 509-
512.
W2. WEIZER, N., AND OPPENHEIMER, G. Virtual
memory management i n a paging environ*
ment . Proc. AFIPS 1969 Spring Joi nt Corn-
put . Conf., Vol. 34, p. 234.
W3. WILKES, M. V. Slave memories and dy
namic storage allocation. IEEE Trans. ~ [
15 (April 1965), 270-271.
W4. - - - - . Time Sharing Computer Systems.
American Elsevier, New York, 1958.
W5. - - . Computers t hen and now. J. ACM 15,
1 (Jan. 1968), 1-7.
W6. . A model for core space allocation i n a
time sharing system. Proe. AFI PS 1969
Spring Joi nt Comput. Conf., Vol. 34, pp.
265-271.
W7. WOLMAN, E. A fixed opt i mum cell-size for
records of various lengths. J. ACM 12, 1
(Jan 1965), 53-70.
Computing Surveys, Vol. 2, No. 3, September 1970

You might also like