In other words, it was similar to what they devised on "Les Mis" for the live recording of the actors singing on set while listening to a pianist perform the music in a nearby booth.
"The theme of visual effects in the movie was tied to the art direction," suggests Sony's VFX supervisor Scott Stokdyk (who worked on Raimi's "Spider-Man" trilogy). "We decided to shoot on set and get in camera art directed pieces of photography. That rippled through production and post, so as animators [under the supervision of Troy Saliba] worked on China Girl, they looked at the marionette performance on set along with the video reference of Joey King's face. Then they looked at what Oz was doing. It was all performance-based reference. For Finley, it was Braff's facial performance and real reference of a capuchin monkey and video reference of real birds since he's a [cross between a monkey and a bird]."
The capuchin monkey proved inspirational. Early on Stokdyk looked at expressive photographs of capuchins with tight wrinkling above the brow that looked almost human. It provided an instant read of his expressions.
The 18-inch China Girl was designed by Michael Kutsche ("Alice in Wonderland") but proved an animation challenge in making the porcelain surface convincing. They wanted to take advantage of the fault lines in ceramics called crazing."We dialed that in on different levels of the face and the body to give it complexity," Stokdyk continues. "Then we had a debate about the clothing. First we talked about a hard surface like overturned saucers that would bounce around and give a simpler movement but that proved too restrictive. Instead, we decided on soft-like doll clothing that allowed her to move more freely while lending more empathy to her. Then we had to be careful that the surface didn't appear too rubbery. The trick was to hide the movements in the face on a cut or a head turn. But the goal was to read the expressiveness through face shapes. It gave us a nice design restriction that we imposed on ourselves to elevate the character into something a lot more interesting."
Indeed, Finley and China Girl recall the Scarecrow and Tin Man, but the difference here is that they actually steal the movie from the wizard and witches because of the strength of their performances. There's no predicting what'll happen when you return to Oz.