Conjoined Twins VFX – Played by Single artist
November 3, 2012 2 Comments
Conjoined Twins – Maattrraan
“Maattrraan” is a Indian(Tamil) feature film based on the life of conjoined twins. This film is directed by K.V Anand and actor Surya played both twins roles. We did around 2000 Vfx shots for this project.
In “Maattrraan” both the characters are joined together at all times, because of which there is interaction in each and every frame. So we decided to go with Body Double with head replacement option with two methods.
Method A : Digital Head replacement
For faster head and body movements, mainly in action and dance sequences, we have used Digital Surya. We scanned 22 facial expressions of Surya in Lightstage, LA. Thanks again to Paul Debevec for helping with the scanning. We have used “Image Metrics” team for facial animation, because of budget constrain, we have used them mainly for wider shots. For closeup shots we redesigned a rig using those 22 scanned shapes. Initially we created morph targets manually based on the scanned expressions, but we could not get a 100% match. Later on Hao Li from LA helped us to create accurate morph targets by warping the rigged mesh with the 22 scanned expressions. To record Surya’s facial performance, we created a simple head rig with a helmet, LED strips, CCD cam and iphone4s. It served our purpose well. Thanks again to Jupiter Jazz team for Skin Shader and Autodesk Maya is our main software.
Method B : Live Head replacement with 5 Camera setup
In this method the main issue was matching Body Double’s camera perspective with Main Artist camera perspective while shooting his head separately in green screen. In order to solve the perspective issues, we decided to capture few extra angles of the Main Artist’s head, so we can match perspective later during in post production. For that we used 3 main cameras and 2 supporting cameras. We shot the Main Artist as 1st twin while a Body Double played the 2nd twin. Later, the Main Artist acted as the 2nd twin in a green screen floor using the 5 camera setup. We used 3D mesh of the Main Artist head, tracked and animated it to match his green screen live head, then projected all 5 camera textures onto that 3d head. The aim was to get the Main Artist’s head (shot in green screen as 2nd Twin) into a 3D environment with live textures, so we can match the Body double’s perspective. This 5 camera projection method is also used for correcting main artist look problems. This method looks simple but it took lot of our R&D time. One of our main task in 5 camera setup was to exactly match 5 cameras position in the 3D environment with the live cameras position. If these cameras are not matched properly, the projection method won’t work at all. To achieve this we used X-box Kinect sensor (like a 3D scanner) to capture camera placements on location. It worked well for indoor shooting, but failed in the bright light outdoor green screen setup. The other important task was to track and match the 3D head with Main Artist’s live head. We visited L.A to scan the lead actor’s expressions using Lightstage. While we were there, Paul Debevec and his team at ICT lab saw our test work and gave us tools to simplify this entire process. It was like image based modeling that creates mesh and cameras based on 5 cameras image data, It helped us tremendously. Thanks to Paul and his team for their help. We have used Nuke to do 3D projection and final comp.
Scene 16 Input
5 Cameras Input
5 Cameras Projection
Scene 16 Output
One more important task was the Main Artist has to match his Body Double’s head and body movements with proper timing. For that we recorded body double’s video while saying 1,2,3,4…. as audio bg track, so Main Artist could observe and remember his Body Double’s head and body movements by using numbers as reference. Because of Surya’s (Main Artist) excellent performing talent, we were able to pull this off without looking for another option. In some situations he has to remembered more that 40 tasks! And lighting wise Dop Soundar has done excellent job in matching both lighting conditions manually, in some situations we have moved lights instead of artist waking. Interestingly in this project we have used almost all type of cameras like Film Arri 435, Arri 3, Digital Cannon 5D, 7D, iphone, Kinect sensor and small CCD camera.
5 Camera studio setup
Apart from these methods, we have used simple head replacement and sometimes full body replacement methods in regular 2D comp too. Overall for 2000 vfx shots we used 8 Indian companies with 400 artists in a period of 2 years time. Below are the links of few final output sequences.
Twins Dance music number Ouput