SC07 Education Program

Desktop to Grid Parallel Session

- CFD
November 10, 2007

Jim Giuliani
Client and Technology Support Manager
Ohio Supercomputer Center
jimg@osc.edu
Agenda

Introduction to Fluid Dynamics

Computational Fluid Dynamics Motivation & Processes

State of the Art in CFD and its Role in Education

CFD Tools
•Java Applets
•MATLAB Application
•FLUENT

2
Introduction
• A brief introduction will be given to the field of Fluid
Mechanics and the types of physical systems that can
be modeled
• Computational Fluid Dynamics (CFD):
– Why CFD
– How it is applied
– What is the current state of the art
• CFD’s role in Education
– Design Tool for the engineer...doctor…???
– Educational tool for understanding
physical phenomena
“The story behind the simulation”
• Applications will be explored that allow computers to
help convey fundamental concepts in fluid dynamics
that allow for increased knowledge discovery

3
What is Fluid Dynamics
• Roughly defined: Fluid Mechanics is the study of
a system in which a fluid is the working medium
• Other fields
– Solid mechanics
– Thermodynamics
• Statics is the study of
systems at rest
• Dynamics is the study
of systems in motion

4
Classification of Fluid Mechanics
Continuum Fluid Mechanics
• Inviscid
• Viscous
– Laminar
– Turbulent
• Compressible/Incompressible

5
Classification of Fluid Mechanics
Problems are also classified in terms of the number
of dimensions of the problem
• 1-Dimensional
• 2-Dimensional
• 3-Dimensional
Most problems are 3-D,
but analysis based on
fewer dimensions is
often meaningful

6
Differential vs. Integral Approach
Differential Equations
– Equations formulated in terms of infinitesimal control volumes
– Solution of differential equations determines the detailed, point
by point, behavior of the flow
Integral Equations
– System under study does not require a detailed knowledge of
the flow
– Equations formulated in terms of finite systems and control
volumes
– Often easier to treat analytically

7
Navier-Stokes Equations
Du ∂p ∂  2  ∂u ∂v ∂w  ∂   ∂u ∂v  ∂   ∂w ∂u 
ρ = ρg x − +  µ  2 − −  +  µ  +  +  µ  + 
Dt ∂x ∂x  3  ∂x ∂y ∂z  ∂y   ∂y ∂x  ∂z   ∂x ∂z 

Dv ∂p ∂   ∂v ∂u  ∂  2  ∂v ∂u ∂w  ∂   ∂v ∂w 
ρ = ρg y − +  µ  +  +  µ  2 − −  +  µ  + 
Dt ∂y ∂x   ∂x ∂y  ∂y  3  ∂y ∂x ∂z  ∂z   ∂z ∂y 

Dw ∂p ∂   ∂w ∂u  ∂   ∂v ∂w  ∂  2  ∂w ∂u ∂v 
ρ = ρg z − +  µ  +  +  µ  +  +  µ  2 − − 
Dt ∂z ∂x   ∂x ∂z  ∂y   ∂z ∂y  ∂z  3  ∂z ∂x ∂y 

Assumptions/simplifications allow us to reduce complexity

•For an inviscid flow, where we neglect viscosity, we
simplify the equations to the Euler equations
•For an incompressible, constant viscosity flow, the
viscous terms simplify significantly (more applicable to
gasses than fluids)
8
 Why Computational Fluid Dynamics
(CFD)
• Once we have classified our problem, we can derive equations that
describe the fluid system
• Due to complex geometries, or lack of realistic analytical equations, we
use numerical methods to approximate the fluid equations
• CFD uses computers to develop
solutions to fluid mechanic
problems
– Ability to solve more complex
geometries
– Ability investigate regions of the
flow where sensors cannot be
placed

9
The CFD Process
• Once the governing equations for a problem have been
derived, the task of solving them numerically can be started
• A fundamental aspect of CFD is to discretize the spatial
domain into small cells to form a volume mesh or grid
• A suitable algorithm to solve the equations is then applied
• Implicit tech.
solve the eqns.
directly
• Time marching
explicit tech.

10
The CFD Process: Gridding
Limitations

• Only 1.5” gap between belly and ground

• To properly represent flow through any gap, should have at least 5 cells between wall surfaces
• Turbulence modeling requires first layer of cells on road and body to be ~0.5”
– This leaves only ~0.5” for remaining 3 layers; results in very high aspect ratio cells and misrepresented
ground/vehicle boundary layer interaction

Boundary layer cells 0.5”

1.5”
Boundary layer cells 0.5”

11
State of the Art in CFD
• Large companies understand the impact of numerical
simulations on R&D and are leading the charge
– SC07 Masterworks series
• Advances in mesh generation allow complex CAD
geometries to reliably be meshed from solid models
• Node and element counts in the 10 to 100 million range
BASE – 0.34
(Cd = Drag coefficient)

XV1 – 0.21

Stock
12
 State of the Art in CFD
• Many commercial CFD applications now support
distributed memory parallel processing
• Production runs common in the 50 to 500 processor
range
• Leading edge runs in the 1000 to 5000 processor
range
• Example: Virtual wind tunnel
Solution driven by velocity inlets on
domain walls

Flow exits domain through

pressure outlet

13
Roll of CFD in Education
• Fluid mechanics courses are in many undergraduate
curriculum
• Computational techniques, specifically visualization, offer
many educational benefits in helping convey fundamental
fluid mechanic
concepts
• Computer
simulations can
allow some lab
experiments to be
replaced with
on-line tutorials
• CFD courses are
mostly found at the
graduate level
14
Case Studies
• We will consider several fundamental problems in fluid
mechanics and examine how different computational
tools can be used to solved the problem and highlight
characteristics
• Applying the different tools to the same problem will help
compare an contrast the capabilities of the different
utilities
• Flow over a cylinder is a fundamental fluid mechanics
problem of practical importance. The flow field over the
cylinder is symmetric at low values of Reynolds number.
As the Reynolds number increases, flow begins to
separate behind the cylinder causing vortex shedding
which is an unsteady phenomenon (and a COOL
visualization!)

15
CFD Tools Overview
Three tools will be discussed in the workshop
• Java applications and Java applets
– Programs written in the Java programming language and
often integrated with on-line web pages
• MATLAB
– Commercial software package for general mathematical
modeling
• FLUENT
– FlowLab (educational product) integrates commercial mesh
generator (GAMBIT) and solver (FLUENT) with simplified
GUIs for students

While only scratching the surface, these applications

cover the range of simulation ability, from simple Java
app, to cutting edge parallel CFD solver

16
Java Applets
Java is a very useful tool for education
Benefits
– Applets allow for easy integration into web based lecture
material and on line exercises.
– Interactive and often include graphics
– Excellent portability and distribution
– Many good applets already exist (some funded by NSF)

Drawbacks
– High degree of programming effort
– Java is not fast enough for many computational tasks

17
Java Applets - Example
engAPPLETS
– NSF funded project at Virginia Tech
http://www.engapplets.vt.edu/

– Fluid Dynamics
– Statics
– Dynamics
Applications that automate
calculations with data
visualization allow
students to interact with
science, not just observe
trends

18
Java Applets – Ideal Flow Machine
Ideal Flow Machine
– Ideal Flow Machine is designed for students learning the
foundations of fluid mechanics
– The term ‘ideal flow' describes the way in which a fluid (liquid
or gas) moves when the effects of compressibility and
viscosity are negligible
– Removing these terms from the Navier-Stokes equations
reduces the problem to one that can be solved analytically or
numerically
• Go to the engAPPLET web page
• Click on Ideal Flow Machine, which is in the left hand
column
• Click on “Launch Ideal Flow Machine”

19
 Java Applets – Ideal Flow Machine

We will use
the following
options from
the pull
down menu
to examine
flow around
a cylinder:
•Free stream
•Circle
•Draw
Streamline
•Vortex 20
Java Applets – Ideal Flow Machine
• Choose the Free Stream option from the pull down
menu
• Type in 10.0 in the Strength input box
• Click anywhere on the mesh. An arrow will appear
on the left hand side showing the free stream
• Choose the Circle option from the pull down menu
• Click on one of the cross hairs and hold down the
mouse button
• Pull the mouse to the neighboring cross hare and
release
• You should see a green circle appear on the mesh

21
Java Applets – Ideal Flow Machine
• Choose the “Draw Streamline” option from the pull down
menu
• Click anywhere
on the mesh
and you will
see a
streamline
from that point

22
Java Applets – Ideal Flow Machine
• Choose Vortex from the pull down menu and
click anywhere on the mesh to add in a vortex
(flow circulating around a point)
• Click on Draw Streamline to visualize the
new flow
• Click on New Flow
and redo the
exercise but with
10.0 strength
and 10.0 angl

23
Java Applets - Example
• NASA Glenn Research Center – “The Beginner’s
Guide to Aeronautics”
http://www.grc.nasa.gov/WWW/K-12/airplane/
• Java applications for:
– Aerodynamics
• FoilSim Interactive Simulator (very good)
• CurveBall Interactive Simulator
• Atmosphere Modeler
– Propulsion
– Hypersonics
– Model Rockets

24
FoilSim Choose between ideal flow
and flow with separation

Sliders allow students to

dynamically change the
orientation and design of the
airfoil and see real time 25
 Curveball
•Extends
the ideal
flow
simulator
by adding
viscosity
•Provides
real world
connection
(i.e.
viscosity
makes
Students to change spin of ball and see
curve balls
quantitative
possible) (lbs of force, change in streamlines)
as well as qualitative results
26 (ball / strike)
MATLAB
Benefits
• Graphical User Interfaces can be designed to lead
students through exercises
• Good quality and easy to implement graphics
• Software can be used for other academic purposes
Drawbacks
• Not as fast as native code
• Software has a financial cost to students

27
Navier2d – Example MATLAB CFD
Program
• Navier2d is a set of MATLAB functions designed to
simulate the motion of incompressible fluids via
numerical solution of the 2D, unsteady Navier-Stokes
equations
• Available from the MATLAB Central file exchange
– Go to http://www.mathworks.com
– Click on User Community
– Type Navier2d in the search box and click Search
• Solver reads in mesh, which allows flexibility to solve
different problems
• Mesh generator is available in a separate MATLAB
program

28
Navier2d
• Graphical User Interface designed into the
application to lead the student through the
different steps of the solution
– Allows for easy step-by-step example problems
– Separate mesh generator allow students to go
beyond the “caned” examples
• Allows students to gain experiences with the
terminology and work flow of computational
fluid dynamics, while still learning about the
flow problem at hand

29
Navier2d – How to Run
• Download the zip file and unpack on your local
machine
• Start up MATLAB and in the Current Directory
window, move to the
directory where
Navier2d resides
• Right click on
Navier2d.m and
click Run
• You should see the
applications main
menu

30
 Navier2d - Mesh
Rather than build a mesh from scratch, we will use a pre-
defined mesh of a cylinder in a free stream
• In the “Mesh Options”
window, click Load
• In the file browser
that pops up, select
cylinder.mat
and Open
• Click View to see
the mesh

31
Navier2d – Boundary Conditions
• To set boundary conditions, click the
Velocity/Pressure button in the
Boundary Conditions window
• To set boundary conditions, click the Select
button
• With your mouse, drag a box around the
nodes you want to select and then press the
right mouse button when done
• With the nodes highlighted in red, click the
Set button to apply the boundary condition to
those nodes.

32
 Navier2d – Boundary Conditions
• For this example, we will choose the following initial
and boundary conditions
– Inlet (left hand side)
• Velocity B/C
• U=5m/s, V=0m/s
– Top and bottom
• Velocity B/C
• U=5m/s, V=0m/s
– Outlet (right side)
• Outflow
(extrapolated)
– Surface of cylinder
• Velocity B/C
• U=0m/s, V=0m/s
• Click File->Close
when done

33
Navier2d – Initial and Solver
Settings
• Under Integration Settings, click on Initial and
set the U velocity to the initial value of 5m/s
• Under Integration Settings, click on Set to set
solver options
• Accept defaults, but change Maximum number of
steps to 2000 and then click OK
• Click Run to begin the solution
• Watch the vortex structure form as the simulation
progresses

34
Navier2d - Results

35
FLUENT
• One of the leading CFD
applications for research
and product development
• A wide range of physical
systems and processes can
be simulated using numerous
techniques and equations
– Compressible/Incompressible
– Combustion
– Porous media
– Flow with heat transfer
– Adaptive mesh refinement
– Parallel solver
36
FLUENT Capabilities

Simulation of a 1
gallon milk
container being
formed by blow
molding.
Simulation
includes the
arison extrusion,
pinch-off, mold
closing and
inflation steps

37
FlowLab – Academic Interface for
Fluent
• Integrates the following three steps:
– Preprocessor
• Meshing the object or domain to be analyzed
• Applying boundary conditions
• Set application/solver control parameters
• Usually graphic intensive
– Solver
• Read in the mesh and supporting input files and solve
• Longest step, usually CPU, memory and disk intensive
– Post Processor
• Read in solution and visually display data
• X – Y plots, histograms, contour plots, animations

38
FlowLab
• Provides a virtual fluids lab for teaching and virtually
reinforcing concepts in fluid flow and heat transfer
• Acquaints students with the following:
– Terminology of CFD
– Basics of geometry and mesh generation
– Setting up a problem
– Concepts of accuracy and convergence
– Post processing
• Several fluid mechanics text books include FlowLab
modules
http://flowlab.fluent.com/collaborations/index.htm

39
FLUENT – High Performance Computing
Example
• Now that we have seen examples of CFD on the
desktop, let’s examine a larger application
• Motivation
– Examine more complex flow systems
– Experience the full modeling process
• Pre-processing
• Solving
• Post-processing
– Introduce the concept of parallel processing
• Solve models quicker
• Solve bigger models

40
 o *
Mixing Problem in 90 Elbow
o
A cold fluid at 20 C flows
into the pipe through a
large inlet, and mixes
with a warmer fluid at
40oC that enters through
a smaller inlet located at
the elbow
• Heat transfer, 3-D,
Turbulent flow
• Will run in parallel on
4 processors

* Fluent 6.3 Tutorial Guide

41
FLUENT – High Performance Computing
Example

• Workshop accounts will be

provided for access to Ohio
Supercomputer Centers
IBM 1350 Opteron cluster
• Due to network load, slow graphic
performance is very possible
1. Open up a CYGWIN terminal
2. SSH to the Opteron cluster:

ssh –X wrk###@opt-login1.osc.edu

where you will replace ### with the number given to you

42
First Steps – Starting a Remote
Application
• To run parallel FLUENT, we are going to request 1 machine
and all 4 processors within that machine
• A graphics window will return after we start FLUENT
– It is possible to do perform tasks through menu interface
– For speed, we will type in most commands
• To request 4 processors
qsub –I –V –l walltime=1:00:00 –l nodes=1:ppn=4
• To start FLUENT In this exercise,
text in a bubble as
module load fluent seen here are
rm -f pnodes commands you will
type into the
cat $PBS_NODEFILE | sort > pnodes FLUENT command
setenv ncpus `cat pnodes | wc -l` window.
fluent 3d -t$ncpus -pinfiniband.ofed -cnf=pnodes

43
FLUENT Command Window

44
 Read in Model and Partition
parallel partition auto use-case-file-method
yes
rc elbow3.cas.gz
print-active-partitions
Solve initialize initialize-flow

45
Display Parallel Decomposition
(NOTE: These are mouse/keyboard commands)
• Click on Display
• Click on Contours
• Change “Contours of” selection
to “Cell Info…”
• Chose “Active Cell Partition”
from the box below
• Change “Levels” to 4, which
is the number of processors
we are using for this run
• Choose “symmetry” from
the “Surfaces” selection
• Click Display
• Click Close

46
Parallel Decomposition

47
Set Initial Conditions
(NOTE: These are mouse/keyboard commands)
Type command:
file write-case elbow4
Click on Solve
Click on Initialize
Click on Initialize
Choose “velicity-inlet-5”
from “Compute From”
Change “Y velocity” to 0
Click INIT
Click OK
Click CLOSE

48
Solve the Model
solve iterate 200
(when finished)
file write-data elbow5
parallel timer usage

49
Display Results
(NOTE: These are mouse/keyboard commands)
Click on Display
Click on Contours
Change “Contours of” selection
to “Custom Field Functions…”
Chose “dynamic-head”
from the box below
Change “Levels” to 80
Choose “symmetry” from
the “Surfaces” selection
Click Display
Click Close

50
Simulation Results

51
FLUENT Observations
• Many problems have static solutions
• Students not familiar with Fluid Mechanics seem to like the
interactive process and obtaining the solution, as long as
step-by-step instructions are given
• Students with Fluid Mechanic behavior will benefit from
being able to interact with solution results and see system
characteristics that have been covered in lecture
• High school students can handle geometry design and
meshing, along with setting initial and boundary conditions
• Junior undergraduate students are able to develop and
solve complete models

52