1

2
3
Very complex rendering: Dynamic sweat and damage, animated trails. Dynamic
wrinkles, muscle maps. Subsurface scattering (via UV-space blur in NIS). SSAO.
Extremely dense models (sub-pixel triangles...). Almost used all the texture samplers on
the skin. 3d Crowd with facial animation. 3d Cloth / softbody for impacts on the face...
See GDC 2010 Fight Night 4 presentation...

4
5
6
Tools that help tuning:
In game image viewing/filtering tools
Photoshop scripts
Debug shaders
Debug/Reviewing environments (lighting)
Artists need to be able to isolate issues, not judge only the final result.

7
Aliasing ... Try to crank up resolution and AA on pc games, see how much more realistic
they look!

8
Photorealistic scenes can be found across generations...

From top left, clockwise: Half-Life 2, Call of duty 4 Modern Warfare, Grand Theft Auto
4, Left for dead, Call of Juarez – Bound in blood, Tourist Trophy (PS2), Gran Turismo 4
(PS2)

Most screenshots are from “Dead End Thrills”

This is a selection of “older” games but I’m cheating here: it’s all about environments.
Humans are harder.

Videogame photography links:

http://enwandrews.tumblr.com/
http://shotbyrobert.com/wordpress/?page_id=1011
http://deadendthrills.com/

9
Some big changes are still perceptually believable. The black and white image also has
more global and local contrast. Global blurring does not change our ability of
recognizing the “human” quality of the face (but blurring only the skin detail while
retaining the edges would for example!)

Most of the visual quality are relative. The lips in the bottom-left picture appear to be
redder than the one on the top, even if only the skin around them has a slight cool tint,
the lips did not change.

Also some “minor” changes in absolute values can make huge differences. The bottom-
mid image has a bit of green added to the midtones, the bottom-right one only
changed the reds making them more pink.

10
We need photorealistic visual targets – photos, collages, high-quality renderings.
Concept art won’t cut it, as it’s good to evaluate some aspects (volume, composition)
but not all of what’s needed to generate a photorealistic image (colour, texture, details)

11
12
13
Possible solutions:
- Re-compute normals from skinned geometry
It’s tricky as the geometry in game can be in a different format and use different rules
than the original in your DCC 3d modeling package. I.E. Quads vs Triangles, subdivision
surfaces, projection of the normals from a high-res model into low-res etc...

-Compute an extended set of weights for normal skinning.

Let's say we derive face normals by averaging the vertices of a face. And that we
compute vertex normals by some form of averaging of the faces of a vertex (usually,
weighted by the areas). Let's assume that the face areas do not change under skinning.
In that case we can compute a set of bone weights and indices that is the average of
the bone weights and indices that act upon the faces of a given vertex.

14
15
16
17
18
19
Shaders are some sort of black box. They model a given function (BRDF, or better, a
given approximation of the whole rendering equation), and each parameter can change
part of the function’s shape.

The art-direction and visual targets define an “objective function”, that we don’t know
mathematically in any form. Artists are “optimizers”, they try to find the parameters
that best fit the shader function to the art-direction.

More parameters, more degrees of freedom = harder work for the artists. We need to
guide the artists, provide intuitive models, limit parameter ranges and dependencies.
Math and physics guides our process, they make us understand what to add and why, if
a parameter makes sense or not, what is the root cause of a given visual defect.

20
Adding more parameters/hacks
-- Increases complexity. Over time shaders become unwieldly, it becomes hard to
understand what causes what.
-- Feedback loop / a problem that makes itself worse. The more complex the shader,
the less it’s understood, the more we will need to add ad-hoc hacks when things go
wrong.
-- Performance problems
-- Makes changes harder: it becomes actually HARDER to art-direct, it becomes harder
to iterate and tune (NAIL THE LOOK).

Everyone is shifting towards less parameters, more physically based models. Many talks
at Siggraph2010 were about GI in movies and its advantages

...It would be a nice project to create an automatic shader-tuner, that fits parameters to
a given reference image. The right comparison metrices are needed, and there is no
hope to make it work over all the dimensions of a typical shader parameter space, but
it could be possible for example to fit SH lighting having fixed the material parameters
for example.

21
22
23
Not only less parameters, coarser granularity!
Material parameters: Skin in FN4 was specified mostly per-skintone, sometimes per boxer
Skin in FNC is mostly tuned globally, only 2-3 parameters per skintone, none per boxer

Lighting parameters: FN4 used 3-4 lighting cubemaps per venue, FNC only one
FN4 tuned lighting per scenario (i.e. Entrances vs In-Game), FNC practically doesn’t.
Cubemaps are SLOW to iterate on!

Artist’s hacks: bending the parameters (the shader “shape”) too much, trying to fix a given
visual problem with the wrong controls. “Encourages” communication: if a given visual problem
is not fixable within the current parameter ranges, talk to an engineer 

Example of “artist friendly” parameters: Normalize specular so that changing exponent does
not change the intensity
It can also make some computation faster, i.e. Translate brightness/contrast parameters in a
single mAdd

Parameter ranges: Avoid having artists “pushing” parameters to levels that do not make sense.
If they feel that need either there is a misunderstanding on how the lighting works, or the
lighting is not working correctly!

Texture parameter “space”: Gamma, different encodings and compression settings.

Author textures in 16bit, find the best representation in the pipeline. Author texture
separately, merge and mix them in the pipeline (i.e. Multiple grayscale texture merged in
different channels of a single in-game one, AO maps pre-multiplied into diffuse and specular
etc...)

24
25
26
27
28
29
30
Bad math in the specular: We tinted specular using diffuse lighting color, this actually
decreased hue variety.

Standard diffuse does not work. In FN4, in-game we used an ambient map (redness) to
simulate SSS. During the NIS this map was replaced by a dynamically generated one
(UV-space blur) This solution can’t work with more varied lighting setups (FN4 is almost
always strongly top-lit). Also the map was not entirely reasonable the way it was
authored.

Sidenote: Noise is good to break highlights, if we wanted to simulate peach-fuzz it

would be good to add a bit of noise in the specular at grazing angles

31
Don’t judge too much:
Don’t let your experience hinder your
immagination
Not the time to be strictly practical
Tinker, Try, Hack, Read, Try
Look at other medias
After this, all the walls around my desk
were cluttered with papers, screenshots,
ideas

32
33
Performance problems:
We need to compute diffuse lighting twice, first for SSS, then for the direct component
Problems if we need to render many characters. FN4 is the ideal case, in game we have
only two boxers

Screenshots: Jensen’s multipole SSS model, D’Eon-Luebke realtime implementation

34
35
36
It’s possible to use curvature to approximate the geometrical neighborhood. We didn’t
do that in our test.

Similar to: “Curvature-Dependent Local Illumination Approximation for Translucent

Materials” Kubo-Hariu-Wemler-Morishima
And “Curvature-Dependent Reflectance Function for Interactive Rendering of
Subsurface Scattering”
“Curvature-based shading of translucent materials, such as human skin”

The most recent and complete effort is “Pre-Integrated Skin Shading” by Eric Penner:
http://www.ericpenner.net/portfolio/

37
38
39
Avoid blurring: Weight occlusion based on sample distance. Line sampling.

Bent normals: did not make a big enough difference for us, rencently Crysis2 shipped
with it

Consider normals:
Sample in 3D -> Orient sampling hemisphere along surface normals
Sample in 2D -> Consider distance between normal-plane and sampled depth.

Specular occlusion:
sample on a line towards the specular reflection. Useful especially on the rim (fresnel
angles)!

40
41
42
Power after cube fetch
Wrong: can’t really change the exponent, only changes contrast. Dependant on the
HDR range of the cubemap.
Such contrast adjustments shift colors (saturation)!

43
Encoding multiple phong exponents in the mips
360 and PS3 (and older PC cards) do not filter across cube faces -> ATI Cubemapgen
library solves this by making sure that the borders of the cube faces match

Your DXT compressor has to be aware of this too, use the same reference colors for
blocks across the seams! Cubemapgen does not does that

44
We like phong because can be baked in cubemaps!

45
46
47
48
49
50
51
We use a few “ubershaders” with static defines for features, so it’s easy to produce
specializations

52
53
54
55
Faster iteration is better than good technology.
I don’t think this applies to only to artists! I think any programmer experienced how
much more important is to have fast iteration when working with shaders. We don’t
have great debugging tools for them (compared to other languages) but being able to
hotswap them easily and to have an immediate visual feedback is usually better than
having great tools but slow iteration. The same applies to scripting languages.
I’ve even found myself more prone to explore an idea with a lesser potential if it was
implementable within the boundaries of what I could iterate fast, than better ideas that
required slower changes.

For FN4 most of our lighters time was spent tuning lighting cubemaps (80%) in
photoshop
FNC: drastically reduced the number of cubemaps, adopted an in-house tool for their
autoring

Another big pain were expression-driven corrective bones in Maya, we have many of
them. Expressions written in MEL were hand-translated in C++ code! We employed a
new node-based scripting engine for these, with a runtime in game and in Maya.

Ownership:
The more “distance” there is between artists and the game (mainline), the less
ownership is possible.
Outsourcing works too. The artists responsible for a given outsourcer does also
effectively own their external changes, test them and check them into mainline

56
57
58
We added a fairly complex motion blur to Champion. It’s does not only blur camera
movements, but also skinned actor ones. It’s also capable of blurring outside the boxer
silouhette, and it’s fairly robust.

We did some tests very early in prepro using optical flow on a filmed version of the
game to quickly add the effect and we preferred that. Later on we retested our
hypotesis using an actual prototype in game and blind testing.

Most of the pre-pro was still done thinking about techniques that could work at 60hz.
This was not a huge bottleneck as our worst case performance in FN4 and in FNC turns
out to be the in-game NIS, that even in FN4 ran at 30hz. In-game FNC could still run at
60hz even if it wasn’t optimized for it.

SSDO could still be a good technique. It requires better tools, light bleeding could be
solved in the future with more accurate ray-casting, importance sampling according to
the cubemap etc...

Some examples of effects driven by the analytic main light: Ear translucency can be
faked with a cheap trick, we boosted specular cubemaps with it and so on..

Again, accurate occlusion is Very important for shapes and volumes -> likeness

59
Hybrid Point/Directional lights are nice. FN4 flashes were a product of both IIRC
(dot(N,D)*dot(N,P)), works nicely.

60
61
62
Having an analytic light gets us INFORMATION, we can use it to drive many more
effects. Ear fake translucency is just a light set opposite to the main and masked with an
ears mask…

63
It’s easy to break the rendering realism by having visible artifacts, especially aliasing or
upsampling defects...

64
65
66
67
68
69
Note: FNC SSAO occludes specular, main light diffuse (using min(shadow,ao)) and
ambient. It’s used as a shadowing component, not only as ambient occlusion.

70
71
72
73
74
75