140 9.AutomaticDynamicStereoscopic3D
However, any object in 3D that draws in front of the focal plane at that location
acts like a window violation because the UI is (typically) rendered last and drawn
over that object. The sort order feels very wrong, as if a chunk of the object was
carved out and a billboard was placed in space right at the focal plane. And since
the parts of the near object that are occluded differ, the occluded edges appear
like window violations over the UI. All in all, the experience is very disruptive.
The two other alternatives are to always render the UI in the world, or to
move the UI distance based on the nearest pixel. In short, neither works well.
Rendering a UI in world space can look good if the UI is composed of 3D ob-
jects, but it can be occluded by things in the world coming between the eye and
the UI. This can be a problem for gameplay and may irritate users. The second
alternative, moving the UI dynamically, creates tremendous eye strain for the
player because the UI is constantly flying forward to stay in front of other ob-
jects. This is very distracting, and it causes tremendous ghosting artifacts on cur-
rent LCD monitors.
The best way to handle subtitles and UIs is to remove them as much as pos-
sible from gameplay or ensure that the action where they are present exists en-
tirely in positive parallax (farther away than the monitor).
9.3DynamicControls
For purposes of this discussion, the two parameters that can be varied are paral-
lax separation and focal distance. The parallax separation, sometimes referred to
as distortion, is the degree of separation between the left and right stereo camer-
as, where a value of zero presents the viewer with a 2D image, and a value of one
is a maximally stereo-separated image (i.e., the outside frustum planes are paral-
lel with the view vector). Obviously, the higher this parameter is set, the more
separated the renderings are, and the more powerful the 3D effect.
The focal distance is the distance between the rendered eye point and the
monitor surface where left and right images converge perfectly. Anything ren-
dered at this depth appears to the viewer to be exactly on the plane of the moni-
tor. A convenient method for checking the focal plane in an S3D LCD using
shutter glasses is to remove the glasses—an object at focal distance appears com-
pletely solid on screen, while objects in front or behind the focal plane have in-
creasingly wide double-images as their depth deviates from the focal plane.
These are not truly independent variables, however. To the viewer, the
strength of the effect depends on several factors, most importantly, where the
nearest objects are in the scene. Understanding the focal distance intuitively is
important. Visualize a large box that surrounds the entire rendered scene. Mental-
9.4ASimpleDynamicS3DCamera 141
ly place this box around your monitor. By decreasing the focal distance, we slide
the box farther away from the viewer, which causes more objects in the scene to
appear behind the focal plane (the monitor) and fewer to appear in front. By in-
creasing the focal distance, we slide the box toward the viewer and bring more
objects in front of the focal plane (the monitor).
9.4ASimpleDynamicS3DCamera
The first method with which we experimented was a rudimentary system that
recognized the importance of the player’s character. It has flaws, but it helped us
discover how important it is to understand the concept of the “comfortable view-
ing zone” surrounding the focal plane, and it’s easy to look at without inducing
fatigue.
CameraTargetTracking
The most obvious method to try was to set the focal distance to the view-space
distance to the actor that the camera is tracking. This makes sense because most
of the time, the player is watching the character that the camera is watching.
Keeping that actor at a comfortable viewing distance (at the plane of the TV) re-
duces convergence fatigue—even when making “jump cuts” to different cameras,
the player does not need to refocus. It was readily apparent that this is not suffi-
cient by itself. In ICO, the actor can sometimes be far from the camera, and other
times be very close. If the parallax separation is held constant, the degree of ste-
reo deviation that objects in the foreground and background undergo can be ex-
treme and tiring to watch, particularly if something else comes into frame near
the camera.
DynamicParallaxSeparation
The next step was to control the parallax separation parameter based on the focal
distance. Based on our experimentation, we found that the usable range of paral-
lax separation values fell between 0.010 and 0.350, depending on the content of
the scene and the focal distance. Although this was an improvement, it was also
quickly recognized that more effort would be necessary to make a truly excellent
S3D experience. However, it was clear that a dynamic parallax separation was
necessary, and it was implemented this way first to better understand the prob-
lem. The following formula was implemented in both ICO and Shadow of the
Colossus initially, with some adjustments to match in-game units:
142
Fi
gu
exc
e
ext
r
Son
y
Par
a
In p
0.350 as
observa
t
allax se
p
room,
w
camera,
detail at
or rock
o
great th
a
er almo
s
pushed
h
in each
g
u
re 9.4. The s
i
e
pt that objec
t
eme window
v
y
Computer E
n
a
llaxSeparat
lain English,
the focal di
s
t
ion that the
c
p
aration to
m
w
hen the cha
r
the parallax
s
large focal d
i
o
r wall com
e
a
t the object s
s
t certainly s
e
h
eavily towa
r
g
ame. See Fi
g
i
mple algorith
m
t
s nearer to t
h
v
iolations are
n
tertainment,
In
ion
=
min(m
a
the paralla
x
s
tance goes f
r
c
haracter, wh
e
m
ake the back
r
acter was fa
r
s
eparation v
a
i
stances. The
s into the fr
a
eems to prot
r
e
es window
v
r
d the viewer,
g
ure 9.4 for a
n
m
always kee
p
h
e camera ap
p
possible, suc
h
In
c.)
a
x((FocalDi
s
4
800.0F,
0
x
separation
v
r
om 300 cm t
o
e
n close to t
h
grounds feel
r
from the c
a
a
lue needed t
o
problem wit
h
a
me from the
r
ude incredib
l
v
iolations.
W
there would
n
example o
f
p
s the characte
r
p
ear disturbin
g
h
as with the
t
9.Automat
i
s
tance -
3
0
0
0
.0F) + 0.0
5
v
aries linearl
y
o 4800 cm.
T
h
e camera, di
d
3D. Howev
e
a
mera and n
o
o
be cranked
h
this setup i
s
side, the pa
r
l
y far out of
t
W
ithout takin
g
inevitably b
e
f
this kind of
a
r
in focus. Ge
n
g
ly close to
t
t
ree on the ri
g
i
cDynamicS
t
0
.0F) /
5
F, 0.35F);
y
between 0
.
T
his was bas
e
d
not need
m
e
r, in a vast
a
o
walls were
up to show
m
s
that anytim
e
r
allax separat
i
t
he TV, and t
h
g
into accoun
t
e
uncomforta
b
a
rtifact.
n
erally, this is
t
he viewer.
W
g
ht. (
I
mage c
o
t
ereoscopic3
.
050 and
e
d on the
m
uch par-
a
nd open
nea
r
the
m
uch 3D
e
a torch
i
on is so
h
e view-
t
objects
b
le areas
pleasing,
W
orse yet,
o
urtesy of
D
9.5Content‐AdaptiveFeedback 143
In the end, we realized that the relationship between focal distance and paral-
lax separation is not simple because for any given bothersome scene, there were
two ideal settings for these parameters. Nearly any scene could look good by ei-
ther significantly reducing the strength of the effect by lowering the parallax sep-
aration value or by moving the focal plane to the nearest object and reducing the
parallax separation value less.
9.5Content‐AdaptiveFeedback
Shadow of the Colossus had more issues with S3D because the camera was free
to roam with the player, whereas in ICO, the camera was provided specific paths
based on the player’s position in the world. This demanded a better solution that
was aware of exactly what the player was seeing and how he would perceive the
depth information presented to him. Figure 9.5 summarizes our observations.
This new algorithm is based on several observations about how people react
to S3D in interactive environments, especially taking into account the areas
where the effect was too strong or too weak to give the correct feeling of gran-
deur or subtlety without causing the player stress. What we noticed was that ob-
jects in the distance were generally acceptable with any parallax separation value,
as long as their stereoscopic deviation “felt reasonable” given the distance the
player would actually cover to reach them in the game. For example, overempha-
sizing the parallax separation could be dizzying at times, especially when stand-
ing on the edge of a short cliff that the player could easily jump down—a heavy-
Figure 9.5. The focal plane, where the TV is, has a range indicated in blue where com-
fortable viewing occurs with minimal stress. The red areas show where discomfort oc-
curs. Notice the tolerance for extremely near objects is very low, whereas distant objects
are acceptable.
TV
144 9.AutomaticDynamicStereoscopic3D
handed use of S3D incorrectly indicated to players that certain areas were peri-
lously high.
The other observation was that objects that come between the focal plane and
the camera with high stereoscopic deviation tend to upset players, especially
when they reach the left or right edges of the screen (window violations). Other
researchers have suggested variable-width black bars on the sides of the screen to
solve the window violation problem [Gunnewiek and Vandewalle 2010], but this
is not an ideal solution. While this works for film, it is not ideal for games. Doing
this causes a loss of effective resolution for the player, still does not address the
issue that objects can feel uncomfortably close at times, and requires tracking all
objects that might be drawn to determine which is closest to the camera and visi-
ble. This is not an operation most games typically do, and it is not a reasonable
operation for world geometry that may be extremely large and sprawling within a
single mesh. The only good way to determine the closest object is to do so by
examining the depth buffer, at which point we recommend a better solution
anyway.
Our approach is to categorize all pixels into three zones: comfortable, far,
and close. Then, we use this information to adjust the focal plane nearer to the
viewer as needed to force more of the pixels that are currently close into a cate-
gory corresponding to farther away. This is accomplished by capturing the depth
buffer from the renderer just before applying post effects or UI displays, then
measuring the distance for each pixel and adding to a zone counter based on its
categorization. Figure 9.6 shows an example of a typical scene with this algo-
rithm selecting the parameters. On the PlayStation 3, this can be done quickly on
SPUs, and it can be done on GPUs for other platforms.
To simplify the construction of this comfortable zone, we define the half-
width of the comfortable zone to extend halfway between the focal distance and
the closest pixel drawn (last frame). This is clamped to the focal distance in case
the closest pixel is farther away. Since the comfortable zone is symmetrical
around the focal plane, the transition between the comfortable zone and far zone
is trivial to compute.
Once categorized, we know the ratios of close pixels to comfortable pixels to
far pixels. We assume all the pixels in the near zone are going to cause some dis-
comfort if they are given any significant stereoscopic deviation. Given the pixel
distribution, we have to react to it by changing S3D parameters to render it better
next frame. (The one-frame delay in setting S3D parameters is unfortunate, but
game logic can be instrumented to identify hard cuts of the camera a priori and
force the S3D parameters back to a nearly 2D state, which are otherwise distress-
ing glitches.)
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.