Professional Documents
Culture Documents
Parallel Programming
Models
Object-oriented
Functional and logic
Data Parallelism
Parallel programs that emphasize concurrent execution of the
same task on different data elements (data-parallel programs)
Task Parallelism
Parallel programs that emphasize the concurrent execution of
different tasks on the same or different data
Task Parallelism
Explicit Parallelism
The programmer specifies directly the activities of the multiple
concurrent threads of control that form a parallel
computation.
Implicit Parallelism
The programmer provides high-level specification of program
behavior.
It is then the responsibility of the compiler or library to
implement this parallelism efficiently and correctly.
Shared Memory
The programmers task is to specify the activities of a set of
processes that communicate by reading and writing shared memory.
Distributed Memory
Processes have only local memory and must use some other
mechanism (e.g., message passing or remote procedure call) to
exchange information.
Bus
Shared memory
Distributed memory
Memory
Network
Implicit parallelism
Implicit parallelism
Asynchronous message-passing: no
synchronization between sender and receiver
processes
Bandwidth
Latency
Ability to overlap communication with computation.
f(x)
0.1 0.2
xi
f(x) 4.0/(1.0+x*x);
main(){
int n,i;
float w,x,sum,pi;
printf(n?\n);
scanf(%d, &n);
w=1.0/n;
sum=0.0;
for (i=1; i<=n; i++){
x=w*(i-0.5);
sum += f(x);
}
pi=w*sum;
printf(%f\n, pi);
}
0.1 0.2
xi
= w f(xi)
f(x) = 4/(1+x2)
n = 10
w = 1/n
xi = w(i-0.5)
Master
W0
W1
W2
Master
W3
Master:
Creates workers
f(x)
0.1 0.2
xi
x
0
0.1 0.2
xi
W0
W1
W2
Master
Master:
Creates workers
Workers:
W0:
Receives local sums from
other workers
Calculates pi
Sends pi to Master
Thread 2
Cache-coherency protocol
Monitors
Locks
Critical sections
Condition variables
Semaphores
Barriers
PThreads
In the UNIX environment a thread:
PThreads
Because threads within the same process share resources:
Two pointers having the same value point to the same data.