lining up 3D geometry to 2D painting

[terryhamel]

I'm working on an amatuer film that has a shot of the actors walking about an ancient stone city. The actors were filmed against a green screen. I brought images of the actors into Photoshop and worked out the horizon line, roughed in form for the buildings, then refined the forms by working out their vanishing points.

I realized (too late?) that working out the scene in 3D could be advantageous and was wondering how people go about setting up a 3D scene with 2D plates of actors.

How do you get the camera to line up with the scene without having a set to measure? [/*:m:wumsclk7]
How would you build the set to scale when there is no scale?[/*:m:wumsclk7]
How do you know how high the camera is off the ground plane when you don't even know the lens size or focal length? [/*:m:wumsclk7]

[s0nkite]

usually from my litle experience they put some marks on the green screen to
track the movements by a 3d tracking programm, now if i had to do this manually i would do it
like this : i would apply a chroma key to remove the green in After effects
and then save it as png seqence with its alpha channel, and then try to match a camera in 3ds max manually. lol, just make it manually

[rockhoppermedia]

Do what we do in forensic photography, very complicated but worthwhile. Work out how tall the lead actor is for example 1 metre 60 cms (divide by 8 for tall man, 7.5 for shortest man 7 for female) do your division sums.

An eighth of the body will be 21 cms. This will be exactly the length of the head. So in photoshop select marquee and make a box around the head. You can use this box underneath a layer to create a grid. This is the clever part you can generally work out the size and height of every day objects like doorways windows and steps. Infact the mathematics involved in working out the head height correlates to shoe size, weight even race. from the height of the head. I have worked with people that can look at a shoe print and work out what the person looks like wether they were carrying something even down to if they are injured. Some banks use for security objects placed in the camera view that are a established unit size so police can work out the dimensions of a criminal.

Now with your lead actors in shot you could work out your sizes. And create a perspective grid set at comfortable unit that you know and understand. For example if i want to measure one metre i just use 5 boxes as a scale. I now this sounds crazy trust me it works.

Rich

Now auditioning for CSI

[rockhoppermedia]

Camera height of the ground plane if it is a straight frontal shot, measure from the actors eyes to horizon line, use pythagoras theorem of the hypotenuse (hoped you paid attention in maths) to work that distance out, SOH CAH TOA> then use that distance to work out camera height. Go and visit the maths department they can work this out in minutes. Take some cookies they dont get out much and nobody invites them to parties. Camera angle if the crew had been switched on they would have taken notes. Insist on this in the future (or between shots you take the neccessary measurements). you can normally work out it out by estimate.

Hope this helps

Rich.

[terryhamel]

Thanks, Rich. I figured there would be some math involved, I just didn't know which math. In my film making self studies, I haven't read how anyone goes about doing this. Shots using math to work out camera angle and set dimensions (painted or real) have certainly been done before such as in SW:AOTC - after Padme and Anakin kiss, the camera rides the buggy with them as they enter the arena. The camera height and angle had to be worked out for the model set. The math had to been extensively used throughout LOTR. I wonder where I can learn more about the "how"..

Since the shot I'm working on was filmed against a green screen, there is no point of reference, but it makes sense to work off an actor's height and eyeline to extrapolate the horizon line, then fabricate the scene's dimensions from there. In my form sketch, I guessed at the horizon line, but knowing how to work it out will most likely result in my being wrong and the painting needing to be reworked. :(

[rockhoppermedia]

Even good old Bill Shakespeare had to rework his plays. I think that with the planning of forms and shapes will be a lot quicker than previous. You have already done the groundwork so the design will have to be changed due to angle of view. But an intelligent guy like you will crack it.

Good luck and I would love to see what you come up with.

Many regards

Rich :wink:

[terryhamel]

This is an overlay composite showing the actor and the background's color, form, outlines and one vanishing point's perspective lines. I've guessed at the horizon line and worked from there, but I'd like to make sure my windows and doors on buildings 1 through 5 are to scale. The actor is roughly 6 feet tall.

http://one.fsphost.com/thpublic/mattepainting/city1-BG-scale-for-actor.jpg

[rockhoppermedia]

I think the perspective looks spot on, the danger here is the raking of the ground plane but i think you may have it solved. The matte looks promising, tattooine prehaps?

the only problem you might find would be any cg elements flying objects being scaled correctly but I think you have it cracked. I would love to see where you go with this.

Regards Rich.

[mordecaidesign]

I think the perspective looks spot on, the danger here is the raking of the ground plane but i think you may have it solved. The matte looks promising, tattooine prehaps?

I agree with Rockhopper, It looks like great perspective. My problem with this shot is the composition and layout. I think you've done an awesome job of creating good composition in your matte painting, but after the action has been composited in it looks like it's going to ruin that composition.

Right now it LOOKS like an actor walking in front of a green screen. Maybe try putting your actor in front of a "tattooine" wall, instead of this open alleyway type setting .

Don't be afraid to scale down the size of your talent footage, sometimes you can achieve alot by showing only a little bit of live footage and alot of enviroment.


Like I said. I like the shot until the talent footage gets in there.

Hope that helps.

[terryhamel]

rockhoppermedia - Thanks for the feedback. It is more of a aztecian city in a lush forest. I've yet to add any greenery such as ivy on the walls or moss on the ground. The city is dotted with sparse decor trees and is surrounded by a massive forest and mountains, but this location is in the middle of the city and I felt it was appropriate to show the bustling actors towered by the city walls. I'm hoping the director will add CG vehicles in the background - that was my thoughts when I was sketching the form. Either way, that's the director's call. Could you explain/rephrase "raking the ground plane"?

mordecaidesign - You might be right about it being the wrong background for the shot, but I won't know until I see the entire footage - I only have screen shots of the actors in key positions to work out the painting. However, I deliberately painted out the other actors in this screenshot so as to contrast the main actor against the background for scale. There are about 5 to 10 more actors moving with/against the main actor. With CG vehicles in the background, it might work. They crew has a great arsenal of CG models already created.

[rockhoppermedia]

Groundplane= floor

Raking placing the floor at an angle to enable false perspective also to show the full figure of an actor. Most theatres have a raking floor which is angled very sublety downwards.

I apologise for the SW reference I have seen enough fan films based on star wars to last me alifetime and I made an assumption sorry dude.

Aztec city i had to do a project on it, the way they used perflectly crafted blocks has confounded modern masons as they used tolerances that are extremely complicated. just a little factoid for you.

I am dying to see this come together

Rich.

[photoshopped]

A very interesting topic, think I will have to have a good read.

[mastermesh]

Basically what's been said is the way to do it... use alphas and things... In some programs, like Newtek's old aura program (they don't sell it anymore, but another company does) you can track your 2d and export the tracks in to 3d, and work with it in there as a guide.... I haven't done a lot of that, but if you ever get in to motioncapture, you can do that sort of stuff all the time. Do some googling for the word rotoscope.

Putting your animation in the background of your 3d program is another route to take. If your program doesn't take animations for background images, you can use a plane in the background and map an avi or whatever on to there as a texture to get your 2d in to 3d and then build based on that....

One thing to remember, when doing all of this is that you can cheat the camera if you know your camera moves ahead of time... and in 3d can more or less set up like they used to do with glass panels in film, just do it with planes and transparency and stuff...

I never will forget on some old Lightwave and trueSpace discussion forums how some folks actually re-did some mc escher paintings in 3d... you have to do a lot of playing with horizon lines and stuff to do that kinda fooling the eyes, but it's highly possible... some of the 3d that came out of that looks nothing like the painting if you view it at a different angle than what they set it up to render at.... can't find a link now, but if you do some googling around you'll find something I'm sure.

[Da_Elf]

i had to do a green screen track and matte once where i had no measurements and i couldnt even get the height of the actor either since from his upper leg down was cut off. the track was quite simple using icarus (from waaaaay back when there was a free copy). i kinda banged my head trying to find a scale reference then remembered they were using electrical tape for the track markers and i just grabbed a role and measured it. helped quite a bit

[terryhamel]

This painting was resigned to the trash bin because I didn't work out perspective correctly and I want to be proficient with perspective before attempting to fix this - or any other painting that is anything but organic.

The problem became apparent when I was testing sketchup to project the painting onto geometry and couldn't get it to take. Frustrated, I tried a photograph but couldn't get it to take either and discovered the lens distorted the image, so I found another software to undistort it and the perspective lined up.

What this taught me was I don't understand perspective theory as well as I thought I did. My problem is knowing how to place the LVP and RVP so a 90 degree plane such as a building wall matches the plate's camera lens. My painting suffered from placing one VP too far/little and subsequently skewing everything, as if it had it's own lens distortion.

After reading several web pages and my old perspective book, I didn't come away any smarter, so I opted to model the city in sketchup and take snapshots for overpainting. I aligned the camera to a 2D person that closely matched the plate, then walked the camera around to find interesting shots. Since there's multiple shots in the city, I can justify the time it took to model the streets (sort of), but this approach isn't practical, especially in a production environment. I'd like to become adequate with perspective, so can anyone refer me to books, articles, sites or projects that teaches it?