"Things that we used to look at 10 years ago as being impossible, we're almost in the next generation of really understanding what it is that we're trying to put up on the screen, and what are the important aspects to look for visually and what resonates with viewers when you're having this experience," explains four-time Oscar winner, Joe Letteri, Weta Digital's senior visual effects supervisor.
There was obviously a lot more to be done in creating twice as many frames, and considerably more time and cost in rendering the images (which director Peter Jackson covered out of his own pocket). As in animation, though, you don't have to render everything in high-resolution all at once.
"The real impact of the higher frame rate is the clarity that it brings to the motion in stereo," Letteri suggests."For us, we didn't need to make too many adjustments other than commandeering it through the production pipeline. Where you needed to be careful was during paint and rig removal because every frame has to be accurate. In terms of animation, some motions can still be done at 24 frames, but eye blinks or foot contacts where you've essentially got sub-frame accuracy to play with, we took advantage of that. Lighting isn't affected by your frame rate other than lightning where you can get strikes on the mark as they would be in the real world."
No finishing tools were required; nothing about high frame rate precludes you from doing anything differently, other than perhaps requiring more lights from cinematographer Andrew Lesnie.
However, the biggest change on "The Hobbit" was in dispensing with miniatures and finessing forced perspective more efficiently. "You're not limited any longer by the size of your miniature and the size of your stage and the size of the camera moves you can do, nor are you limited by production considerations -- you can make whatever changes you want," Letteri adds.
Now forced perspective has to be dynamic, where the stereo camera is moving on a crane rig. So they needed to slave two motion control systems together: one to encode the motion and one to repeat the motion at a different scale. Thus, you have two stages operating simultaneously, one with Ian McKellen on a green screen stage as Gandalf; and one with the dwarves on a dress stage for their scale, seeing the composite live in the cameras so that the camera operators and Jackson can see what's going on and figure out how to get the actors to work together.
"Logistically as well as technically, it gets quite complicated, especially for the actors, who have to keep all this in their heads and imagine the 3-D space in front of them where all these missing actors are going to be placed and making sure they don't walk through them," Letteri explains.
The most dramatic improvement is Gollum (Andy Serkis), who on the surface appears the same (they thought of de-aging him but it didn't look right). Yet he represents the sum of everything Weta has learned about animation. In "Lord of the Rings," they used motion capture but didn't capture Serkis' face until "King Kong"; and on "Avatar" they added the head rig and the virtual camera. They studied more about skin, muscles, eyes, and hair along with light transport, dynamics, and simulation; and on "Rise of the Planet of the Apes," they figured out how to bring all of that to a live-action stage.
"On 'The Hobbit' it all comes home," Letteri notes. "You've got Andy and Martin Freeman in the very first scene that they shot of the movie with the performance capture integrated with the live-action set. It's all being recorded at a higher frame rate of 48 frames; and we can recreate Gollum now with proper physiology for his muscles and bones and eyes and skin and teeth."
Like everything else in "The Hobbit," we gain a greater perception of the goings on in Middle-earth.