A Uniform Geometry Workflow for Cutscenes and Animated Feature Films
Adrien Herubel, Venceslas Biri, Stepane Deverly, Antoine Galbrun
At DuranDuboi, as we started the preproduction of our first animated feature film in 2009 it became clear that our VFX production pipeline was no longer adequate. Instead of small, manageable projects involving a few generalists, we were now dealing with massive amounts of data requiring intense, short-term efforts from multiple teams of specialists. So with a relatively low budget, we focused on developing from scratch an efficient workflow for handling the massive amount of geometry produced by the different production departments.
But manipulating dynamic geometry is costly both in terms of computing power and storage. For example, the complex Autodesk Maya rigs on the main characters take nearly 60ms per frame to evaluate. In a typical shot, two or three of these characters will be present, making the sequence impossible to animate in real-time. Even worse, dynamic cloth and hair simulations can take up to a minute per frame to compute. Therefore, we decided that geometry should be precomputed or baked whenever possible.
Our key idea was to rely on a single geometry cache format tailored for efficient read, plugged into every software package we used, from modeling to compositing and rendering. Our ubiquitous cache system allowed us to design a set of tools for modification and visualization of the files. Those tools were used uniformly across the production pipeline and allowed us to reduce the usage of Maya, especially for automated geometry processing and quality control.
Moreover, this uniform geometry pipeline improved our workflow by allowing multiple departments to work simultaneously and reduced the overall computational and storage costs. As it turned out, we weren’t the only ones thinking along these lines. Sony Pictures Imageworks simultaneously developed the basis of a similar solution, Alembic, which is an open interchange format focused on efficiently storing baked geometry (www.alembic.io). Alembic has now grown into a production-proven solution; it is stable and efficient enough to serve as the exchange format and API to implement the same pipeline as presented here. In this chapter, I will describe the four parts of our cache and show how it is integrated into our workflow.
Technology
Our cache system can roughly be divided into four parts: a low-level binary cache format, an I/O framework, commercial software plug-ins, and tools (see Figure 5-1).
Figure 5-1. The cache system software stack: low-level IO, streaming and serialization framework, software plug-ins, and tools
Cache Format
The bootstrap section contains basic metadata and the header offset. Each header entry contains chunk metadata.
The cache file format is oriented towards two use cases: writing a huge quantity of geometry once, and reading parts of the geometry often. As the format is designed for writing in batches, it will be slightly inefficient in the case of multiple read-write cycles. However, in a production with a complex pipeline, this situation rarely occurs. Instead, each department produces caches for one department and uses cache from another, so the need for read-write caches is almost nonexistent.
Our files contain an arbitrary number of chunks. A chunk is an array of data, defined by a data type, a size, and a tag. A cache can be viewed as a general-purpose binary chunk dictionary. The binary layout on disk is shown in Figure 5-2. The bootstrap is a fixed-size header containing the API version number and an offset to the header. The chunks are written contiguously and an array of chunk metadata is stored, with each metadata cell containing a chunk offset, a size, a type, and a tag.
Figure 5-2. Binary layout of our cache file format
When a cache file is opened, the bootstrap is read to check for compatibility. Then using the header offset, the whole metadata array is read and transformed into a hash map. Chunks are read only on demand.
During the write process, chunks are immediately written to the disk, next to the last chunks of the cache, but metadata is kept in RAM. Metadata is stored only when all of the chunks have been written, and then the bootstrap section is updated with the header offset.
On top of the low-level general-purpose cache format, we built a whole I/O framework capable of storing a full Maya scene in a cache file and streaming data from the cache to memory on demand.
We define an object as an independent component in a 3D scene such as a mesh, a group of hairs, or a particle system. Objects are split into several chunks referenced by a tag nomenclature. Atomicity is important for optimizing memory occupation depending on the use case; therefore a chunk contains only a small indivisible part of an object. The serialization and deserialization routines allow us to store and reconstruct any set of baked objects from the cache.
The basic I/O procedures can be used to access geometry directly, but for reading complete sets and shots, it is important to enforce strict memory occupation constraints. The framework contains a streaming component in charge of loading complete objects on demand and discarding the least recently used objects when memory occupation is too high.
Our cache system is tightly integrated into most of the software packages used in production. The system is flexible enough so that the same cache file can be used for various use cases.
Cache Export
In our production pipeline, Autodesk Maya was used to produce final models and animations; therefore the cache export module was implemented as a Maya plug-in.
Any arbitrary scene hierarchy, or sub-part of a hierarchy, can be baked into a single cache file. We currently support mesh, nurbs, locators, particles, and fur geometry exported from the commercial software Joe Alter Shave and a Haircut. Objects are exported over an arbitrary number of frames into the same cache file. Static attributes, such as mesh topology, and UV coordinates are stored once. Potentially animated attributes such as transform matrices, deformed mesh vertex positions, and normals are stored for each frame.
As previously stated, objects are divided into numerous chunks. For example, the vertices of a mesh at a given frame n are stored in a separate chunk from those for topology, or normals, or UV coordinates. We also store bounding boxes and transforms separately.
Cache Preview
The cache preview offers real-time rendering of the geometry directly in the viewport and plays complex shots interactively. In this mode, the software has no knowledge of the geometry as it is exposed only as OpenGL proxies. Rendering is performed entirely in the plug-in and takes advantage of the streaming framework to load only what is needed to render. Whereas in this mode editing the geometry is impossible, it is faster to load, render, and play than classic Maya geometry and has a minimal memory footprint. Moreover, the on-demand nature of the frameworks enables us to expose only bounding boxes, or points (see Figure 5-3) to further reduce memory occupation on multi-gigabyte sets (see Figure 5-4). The preview mode is also available in The Foundry Nuke where it can be tremendously useful during compositing.
Figure 5-3. Various cache usages for the 15K-polygon horseman model. From top: preview in Autodesk Maya, preview in The Foundry Nuke, instancing, point mode, bounding box mode, procedural in Pixar RenderMan
Figure 5-4. Screen captures of the Autodesk Maya viewport. The scene contains approximately 5M polygons and is displayed using various cache modes. We compare memory usage of cache display modes and Maya geometry. Measured on the City model, using a Core i7 920 2.67GHz, 16GB RAM, 7200tpm HDD, NVIDIA Quadro 4000
Cache Geometry
When we need the geometry to be available in Maya for editing to solve collisions in physical simulations, we provide a cache geometry mode mimicking the geocache feature of Maya, but again, largely faster and with less memory overhead (see Figure 5-5). It involves injecting animated vertex data into an existing topology, making it available for editing and colliding in Maya. Later the edited geometry can be baked again and used as a preview.
Figure 5-5. Playback speed and memory usage for various cache modes against Maya Geometry Cache. Measured on the horseman model, using an Intel Core i7-920 2.67GHz, 16GB RAM, 7200tpm HDD, NVIDIA Quadro 4000
Rendering Caches
The Pixar RenderMan interconnection is a fundamental building block in our uniform geometry pipeline, as a geometry cache can be used directly as a procedural primitive in RenderMan (see Figure 5-3), thus removing the need for geometry in RIB (RenderMan Interface Bytestream Protocol) files. Cache files are smaller and more flexible than RIB files. They can also be up to 10x faster to render than RIB files due to fast deserialization and coherent data layout.
Custom Tools
Finally, we developed a whole ecosystem around this cache format: real-time preview of multiple cache files, merging two cache files, quickly editing content of a chunk, adding interpolated frames, and even injecting additional data. We found that a homegrown system makes it easy to quickly create tools for processing geometric data, greatly enhancing our flexibility.
We found two tools particularly useful in production. First, the real-time preview in a lightweight visualizer was a real productivity gain for conformation and validation purposes. The visualizer could be opened directly from the asset manager and was much faster than using Maya to quickly validate data passing between departments. The merging tool was also a key component, as it enabled teams to work in parallel as described in the next section.
The production of an animated feature film is generally divided into multiple departments. Each department works with specific input data, received from other teams, and specific output data, which is subsequently turned over to other departments (see Figure 5-6).
Figure 5-6. Geometry asset production workflow
Multiple departments produce geometry data, often for the same assets. This strong dependency chain is problematic, especially when retake requests are made. We tried to improve this process by systematically consolidating geometric data exported by departments in cache files. We also automated the merging of data produced in parallel for the same asset.
Our Pipeline
Great pipeline tools are tailored to production needs. Early specification of features and data models allows room for optimization and general efficiency.
Teams
As I mentioned at the beginning of this chapter, full-length films are characterized by high quality expectations and an enormous amount of produced data. Short animated films and small-length VFX productions can be handled entirely by a small team of generalists, but full-length productions call for multiple teams of specialists dedicated to the production of one stage or one particular type of asset.
As an example, our production teams were divided according to the following table. The numbers in Table 5-1 do not look high because they do not reflect the fact that most departments are not at their full size during the whole production time, allowing some artists to work across multiple departments.
Table 5-1. DuranDuboi Animated Feature Film Preproduction Departments and Their Sizes
Department |
Approximate team size |
|---|---|
Modeling |
10 |
Rigging |
5 |
Layout |
10 |
Animation |
40 |
FX |
5 |
Texturing |
10 |
Look development |
10 |
Hair |
5 |
Dynamics (Cloth) |
5 |
Lighting |
10 |
Compositing |
10 |
An asset is a logical entity which represents all the aspects of an atomic component across production stages. Asset types are generally production-specific or pipeline-specific. The most common are Characters, Sets, and Props. Under the name Asset are regrouped all the files, rules, or comments which define an asset, such as references, designs, Maya scenes, animation curves, or shading assignments.
In our pipeline, each of these files is defined by arbitrary properties such as the originating department or the quality setting. Each physical file belonging to this logical association is versioned so that each published version can be validated and each change can be reverted. For example, a Character could be composed of two base mesh files from the modeling department, one high-poly, and one low-poly; a hair model from the fur department; and a rigged model built by the rigging team. Each of those would be versioned separately. Each published file version is conserved and stored as a read-only file.
Artists generally work on “work in progress” files (or wip), which are not attached to a given asset but to an owner and team name. Versioning occurs when the artist publishes a file candidate for validation, generally using a dedicated pipeline tool. Upon publishing, the validation process begins, as well as the publish callbacks.
Each time a file is published or validated on the pipeline, a set of rules is automatically applied according to the file type, the nature of the validation, or the originating department. These callbacks accomplish a wide variety of tasks: validation requests, model merging, turntable generation, or automated conformation and tests. Thousands of these processes are executed each day and dynamically dispatched on the renderfarm. The ability to automate parts of our production pipeline was essential in the development of the geometry workflow.
Parallel Workflow
Our parallel pipeline (see Figure 5-7) automatically produces a cache containing baked geometry each time a new version of a model, an animation, or a simulation is published. Therefore each asset, animated or static, is available in form of a cache file.
Figure 5-7. Geometry asset production workflow using cache merging. Many departments can now work in parallel as merging is automated upon asset publication
The main idea of a parallel workflow is to identify for each department what parts of the scene need to be editable in Maya and what parts can use baked data from caches. The layout team uses only the cache preview mode since they have no need to edit geometry, but must only populate shots with the needed assets. The animation department uses mixed-mode scenes, where animated characters are rigged, and where the set is either in a cache bounding box (which is a cache preview mode with only bounding boxes and matrices in memory) or in cache preview. The FX team in charge of fluid simulation needs partial geometry data for collision handling, so they use both the cache preview and cache geometry modes where necessary. Finally, the shading and lighting departments use only cache preview for shading assignment and light-rig construction.
Our pipeline allows the user to dynamically override the default settings and switch the cache mode of an asset, even back to plain old rigged geometry.
It is to be noted that our pipeline makes heavy use of the renderfarm for all baking and merging operations. The process is entirely transparent to the user, as baking and merging requests are asynchronously dispatched.
Merging
Multiple departments can induce the generation of baked geometry: modeling and texturing teams produce static geometry, cloth artists provide baked simulation, and animators create baked animation. One problem that can result from this workflow is that multiple teams might be working on different parts of the same model simultaneously. To address this problem, our pipeline enables this feature by automating the merging of cache files produced by multiple departments (see Figure 5-8). For example, a character cache in a lighting scene has UV coordinates and topology from the modeling department, animated shapes from the animation department, and animated clothes from the cloth department. If a modeler makes a retake on UV coordinates, a new cache is generated, then merged with previous animation and cloth caches, and finally updated in the lighting scene.
Figure 5-8. Each department produces different types of geometry data. Each time an asset is updated by a department, a cache is exported. Then the caches of various departments are merged together using a set of rules
Conclusion
At DuranDuboi, we developed a framework and a complete set of tools dedicated to the efficient handling of baked geometry in cache files. The system is used uniformly across all of our production pipelines. The ability to allow simultaneous work on the same models by different departments as well as the automated external processing of geometry data have proved extremely useful.
Our geometry production workflow is built to enable multiple teams of specialists to work in parallel. This solution is combined with our other tools such as the production tracker, asset versioning system, and job queue dispatcher. Such a system would remain equally efficient with an order of magnitude more people. Nevertheless, because this pipeline is aimed at tackling big production challenges, it is probably too constrained to suit small-scale teams of generalists.
Our uniform cache system reduces disk usage by consolidating final data into one file per asset and removing the need for geometry in RIB files. The cache preview and cache geometry modes in Autodesk Maya are faster and lighter than the alternatives. The main drawback of our system is the rigor imposed on users. As the systematical locking of various parts of the scene might appear daunting to artists, we addressed this issue by allowing them to switch back to plain geometry whenever needed.
The system has been used in production and will be upgraded. We expect future versions of the system to be able to support more commercial software programs such as VRay and Houdini. We know that the tag system for data chunks could benefit from increased flexibility and plan to address the issue. For example, each chunk should be referenced by more than one tag, and the chunk type metadata could be replaced by a tag. As geometry complexity is rapidly increasing, particular attention should be paid to memory management and out-of-core capabilities, notably using virtual memory mapping approaches such as presented in Kamp.1
________________
1P.H. Kamp. “You’re doing it wrong.” Communications of the ACM 53:7 (2010), 55–59.