Threads versus Events

CSE451
Andrew Whitaker

This Class
Threads

vs. events is an ongoing debate

So,

neat-and-tidy answers arent necessarily

available

Our

goals:

Understand

the debate
Analyze arguments
Form our own opinion

A Brief History of a Debate

1995:

Why Threads are a Bad Idea (for most

purposes)

John Ousterhout, (UC Berkeley, Sun Labs)

2001:

SEDA: An Architecture for WellConditioned, Scalable Internet Services

Staged, Event-driven Architecture

M. Welsh, D. Culler, and Eric Brewer (UC Berkeley)

2003:

Why Events are a Bad Idea (for highconcurrency servers)

R. van Behren, J. Condit, Eric Brewer (UC Berkeley)

Background
Problem:

How do scale up
servers to handle many
simultaneous requests

Server

workloads have a lot

of I/O, a little CPU

HTTP processing
Send HTTP request
over network
Read and parse
HTTP request
Read file from
disk
Invoke write syscall

time
Send response over
network

Quick Example
Q:

Suppose it takes 10 seconds for a web server

to transfer a file to the client. Of this time, 10
milliseconds is dedicated to CPU processing.
How many simultaneous requests do we need to
keep the CPU fully utilized?

A:

1000

Concurrency Strategy #1: Threadper-Request

Run

each web request in its own thread

Assuming

kernel threads, blocking I/O

operations only stall one thread

Review: Web Server Worker Thread

while (true) {
1. Read from socket to extract
request URI (e.g., index.html)
2. Read desired file into a buffer
3. Write file over the socket
4. Close the socket

These are
blocking calls

Concurrency Strategy #2: Eventdriven Execution

Use

a single thread for all requests

Use non-blocking I/O
Replace

blocking I/O with calls that return

immediately
Program is notified about interesting I/O events
This

is philosophically similar to hardware

interrupts
Tell

me when something interesting happens

Event Loop Pseudocode

while (true) {
// Ask the OS for Sockets with active I/O;
// this blocks if no socket is active
Socket sock = getActiveSocket();
if (sock.isReadable())
handleReadEvent(sock);
if (sock.isWriteable())
handleWriteEvent(sock);
}
In a pure event system, this is the only thread

Details: UNIX Select System Call

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
struct timeval *timeout);

Select

takes three arrays of file descriptors

Reads,

writes, and exceptional events

Note that file descriptors may represent an
open file or a socket
Select

returns the subset of each array

that is ready for reading, writing, or has an
exceptional condition
Or,

blocks if no FDs are active

Event Handler Pseudocode

private Hashtable perSocketState;
// this code *never* blocks
public void handleRead(Socket sock) {
// lookup state about this socket
SocketState state = perSocketState.get(sock);
state.bytesRead +=
sock.nonBlockingRead(state.buffer,state.bytesRead);
// if weve read the entire request, send a response
if (state.bytesRead >= REQUEST_LENGTH) {
sendResponse(sock);
}
else {} // do nothing; wait for another event
}

Can You Think of a System that

Works This Way?
GUI

frameworks (like
Swing)
Single

event-handling

thread
User programs install
event listeners to receive
callbacks

Event
Loop

Event Handlers

Why Events?
A potentially
No

Less
A

simpler programming model

concurrency, locks, conditional variables, etc.

memory footprint

single stack (at most, a stack per CPU)

More

scalable?

Cut to Ousterhout

Critiquing Ousterhout
Events
We

dont handle true CPU concurrency

can fix this with an event-loop per processor

But,

then we have multiple threads

Programming

model becomes complex if

event handlers do not run to completion
This

requires stack ripping to maintain state

State Transition Diagram for the Web

Server

Reading
request

In

Reading
File

Writing
File

a threaded system, state transitions are maintained

by the stack
In an event system, we must manually maintain the
state of each request

Stack ripping

Andrews Opinion: Code

Understanding
Event

systems are difficult to read /

understand
Which

events happen in which order?

while (true) {
// Ask the OS for Sockets with active I/O;
// this blocks if no socket is active
Socket sock = getActiveSocket();
if (sock.isReadable())
handleReadEvent(sock);
if (sock.isWriteable())
handleWriteEvent(sock);
}

Threads vs Events in Practice

Both

are used in production systems

Apache

is multi-threaded / multi-process
Lighttpd is event-driven
Which

is better? Why?

The Devil is in the (Implementation)

Details
Many

systems did not support kernel threads

Making events more attractive

Support

for non-blocking I/O is uneven

e.g., UNIX/Linux do not support non-blocking file I/O

The

UNIX select command has poor scalability

(Banga et al, 1998)

But, more recent alternatives to select are much more

scalable (Linux epoll)

Thread

packages often do not scale (Welsh, 2001)

But, optimizations can vastly improve thread scalability

Van Behren 2003, Linux 2.6 and NPTL

So, What Can We Say For Sure?

Event-based

Requires stack ripping

Harder to track control flow
Possible exception: run-to-completion event handlers
(e.g., Java Swing GUIs)

Event

systems use fewer resources

At most a stack per processor

Event

systems are harder to program

systems require less synchronization

But, they still require some on multi-processor systems

(which is every system!)
Open question: is this is any easier to program?

Software Engineering Issues

Usually, choice

of concurrency model is
made for historical reasons
Unfortunately, changing concurrency
models is very difficult
Use
threads
Begin
project
Use
events