KONERU LAKSHMAIAH COLLEGE OF

ENGINEERING
(ATONOMOUS)
Approved by AICTE, Accredited by National Board of Accredition and ISO 9001-2000
certified
Green fields, vaddeswaram, Guntur dist., A.P.India-522 502.
Affiliated to Acharya Nagarjuna University

STRATEGY OF PARALLEL
COMPUTING

PRESENTED BY

NAME: K.NAGA TEJA NAME: K.IMMANUEL

KLCE KLCE
PHNO: 9030686681 PHNO: 9642672002

Email: naga.tejarocks@gmail.com Email:

immanuel.kota@gmail.com
 • Only one instruction may
execute at any moment in
time.
ABSTRACT:
Serial/scaled computing
The very basics of parallel
computing, and is intended for
someone who is just becoming
acquainted with the subject. It
begins with a brief overview,
including concepts and terminology
associated with parallel computing.
The topics of parallel memory Parallelcomputing
architectures and programming
models are then explored. These
topics are followed by a discussion
on a number of issues related to
designing parallel programs.

1. INTRODUCTION

Traditionally, software has been The compute resources can include:

written for serial computation:
• To be run on a single
computer having a single
Central Processing Unit
(CPU);
• A problem is broken into a
discrete series of instructions.
• Instructions are executed one
after another.
• A single computer with
multiple processors;
• An arbitrary number of
computers connected by a
network;
• A combination of both.
savings. Parallel clusters can be
built from cheap, commodity
components. The very basics of
parallel computing, and is intended for
someone who is just becoming
acquainted with the subject. It begins 4.METACOMPUTING
with a brief overview, including
concepts and terminology associated
with parallel computing. The topics of
parallel memory architectures and
programming models are then explored.
These topics are followed by a
discussion on a number of issues related
1.SHARED MEMORY COMPUTER
to designing parallel programs.
MODEL:
DEF: The simultaneous use of more
than one processor or computer to solve
a problem or task is represented as
“PARALLEL COMPUTING”

There are many types of computers

available today, from single processor or
'scalar' computers to machines with
vector processors to massively parallel
computers with thousands of
microprocessors. Each platform has its
own unique characteristics.
Understanding the differences is
important to understanding how best to
program each. However, the real trick is
to try to write programs that will run
reasonably well on a wide range of
computers.
TYPES OF PARALLEL
COMPUTERS:
1.SHARED MEMORY
ARCHITECHTURE
2.DISRIBUTED MEMORY
ARCHITECHTURE
• Shared memory parallel
computers vary widely, but
generally have in common the
ability for all processors to
access all memory as global
address space.
• Multiple processors can operate
independently but share the same
memory resources.
• Changes in a memory location
effected by one processor are
visible to all other processors.
• Shared memory machines can be
divided into two main classes
based upon memory access
times: UMA and NUMA.

3.CLUSTER COMPUTER Advantages:

 characteristic. Distributed
• Global address space provides a memory systems require a
user-friendly programming
communication network to
perspective to memory
• Data sharing between tasks is
connect inter-processor memory.
both fast and uniform due to the
proximity of memory to CPUs

Disadvantages:

• Primary disadvantage is the lack

of scalability between memory
and CPUs. Adding more CPUs
can geometrically increases
• Processors have their own
traffic on the shared memory-
CPU path, and for cache coherent local memory. Memory
systems, geometrically increase addresses in one processor do
traffic associated with not map to another processor,
cache/memory management. so there is no concept of
• Programmer responsibility for global address space across
synchronization constructs that all processors.
insure "correct" access of global
• Because each processor has
memory.
• Expense: it becomes increasingly
its own local memory, it
difficult and expensive to design operates independently.
and produce shared memory Changes it makes to its local
machines with ever increasing memory have no effect on the
numbers of processors. memory of other processors.
Hence, the concept of cache
coherency does not apply.
2.DISTRIBUTED MEMORY
Advantages:
MODEL:
• Memory is scalable with
number of processors.
Like shared memory systems, Increase the number of
distributed memory systems vary processors and the size of
widely but share a common
 memory increases
proportionately.
• Each processor can rapidly
access its own memory
without interference and
without the overhead
incurred with trying to
maintain cache coherency.
Cost effectiveness: can use
commodity, off-the-shelf
processors and networking.
Disadvantages:
• The programmer is
responsible for many of the
details associated with data
communication between
processors.
• It may be difficult to map
existing data structures, based
on global memory, to this
memory organization.
• Non-uniform memory access
(NUMA) times
3.CLUSTER COMPUTERS:
Distributed memory computers can also
be built from scratch using mass
produced PCs and workstations. These
cluster computers are referred to by
many names, from a poor-man's
supercomputer to COWs (clusters of
workstations), and NOWs (networks of
workstations).
They are much cheaper than traditional
MPP systems, and often use the same
processors, but are more difficult to use
since the network capabilities are
currently much lower. Cluster computers
are also usually much smaller, most
often involving fewer than 100
computers. This is in part because the
networking and software infrastructure
for cluster computing is less mature,
making it difficult to make use of very
large systems at this time. Below is a list
of some local clusters and their
characteristics, plus some other notable
systems from around the country.
4.METACOMPUTING:
Metacomputing is a similar idea, but
with loftier goals. Supercomputers that
may be geographically separated can be
combined to run the same program.
However, the goal in metacomputing is
usually to provide very high bandwidths
between the supercomputers so that
these connections do not produce a
bottleneck for the communications.
Scheduling exclusive time on many
supercomputers at the same time can
also pose a problem. This is still an area
of active research.
REAL TIME APPLICATION FOR
PARALLEL COMPUTING:
An IFS TL2047L149 forecast
model takes about 5000 seconds
wall time for a 10 day forecast
using 128 nodes of an IBM
Power6 cluster.
How long would this model take
using a fast PC with sufficient
memory? (e.g. dual core Dell)
ANS: Ans. About 1 year
This PC would also need ~2000
Gbytes of memory
CONCLUSION:
- Have ever increasing processors,
memory, performance, but
- Need more space (new computer
halls = $)
- Need more power (MWs = $)
2.Parallel computers
require/produce a lot of data (I/O)
- Require parallel file systems
(GPFS, Lustre) + archive store
3. Applications need to scale to
increasing numbers of
processors, problems areas are
- Load imbalance, Serial sections,
Global Communications
REFERENCES:
• A Library of Parallel
Algorithms, www-
2.cs.cmu.edu/~scandal/nesl/
algorithms.html
• Internet Parallel Computing
Archive,
wotug.ukc.ac.uk/parallel
• Introduction to Parallel
Computing,
www.llnl.gov/computing/tut
orials/parallel_comp/#Whati
s