General 3D Question - Matching 2D coordinates with a 3D space

OpenGL Development JOGL Development
#1

Hey everyone.

So here’s my pickle. I’ve got a game where we have a 2D background and 3D objects in the foreground. Everything is drawn in Ortho mode. Originally, we had our artist drawing a psuedo-3D perspective that was mathematically easy to determine for us: rotated 45 degrees then “squished” vertically by tan(30). Effectively this creates a rather nice looking fake isometric view which is very easy to put in programmatically. Basically, all logic is done as if everything exists in a simple top-down 2D grid, then the resulting positions are rotated 45 degrees around the center of the playing area, then their Ys are scaled by tan(30). Simple and looks good.

It turned out that we changed our art style and wanted to be able to reuse assets, so the artists are now creating the levels in 3D and then exporting the renders as the new backgrounds. This is a literal 3D perspective, obviously. I know the rotation, height, etc of the camera in the 3D modeling program, but that doesn’t make it any easier to figure out how to programmatically translate a 2D grid to match the perspective. I’ve tried scaling based on distance from the camera, but I’m having trouble matching it exactly.

My question:
Is there any simple way to get OpenGL to do this for me? If not, any ideas on how I can get this implemented? I just feel like this exact thing must have been done already before (like in FFVII, sort of), so it can’t be rocket science.

Thanks a lot, everyone.

Eli

#2

I don’t really understand what you are trying to do, could I get a picture?

#3

do you try to get 3D position from a 2D rendered image ? if so you cant. The only thing you can get is “a ray” (as there is not only one solution), perspective projection is not reversible if you dont know z.

in another hand there is a very efficient technic to retrieve 2d texture coordinate (btw can be a grid) from arbitrary 3d rendered image using a reference polygon (let me know if this is what you’re looking for) => for example : it is used to re-texture photo / change material / etc…

#4

Okay, let me be more specific.

I know the coordinates of the camera in the 3D world:

Translate -55.097 50.836 55.111 (xyz). Rotate -35.00 -45.00 0.00 (xyz)

The problem is this:

  1. I have an image that is the rendered version of 3D space drawn behind everything else as a 2D sprite (in ortho mode, where one unit represents one pixel).
  2. I have some 3D models that are from the same 3D space, but are literally drawn as 3D, also in ortho mode. They’re drawn on top of the image.
  3. The models programmatically are stored as if they exist in a simple 2D grid, but must be drawn via rotations, translations, and scales so that their position is in the right location in the 3D world.

So say (0,0) in the grid is positioned at the very top center of the image (from the 45 degree rotation). That means that (length-1, length-1) in the grid is in the bottom center of the image, (0,length-1) is at the left center, and (length-1,0) is the right center. The complexity comes in when I need to decide exactly where the in betweens lie, and the scales that I need to give them to match the background.

Does that make any sense?

#5

Not 100% sure what you mean by “ortho mode” - do you mean that there’s no perspective transform?

Also the listed co-ordinates of the camera in the 3D world need more detail. There are various ways to combine them, and you’ll need to find out what the software your artists used did. Really you want to get the whole transform as a single matrix, because that is fairly unambiguous (the only thing left to assume/specify is whether it’s for pre- or post-multiplication).

Finally, if I understand correctly what you’re doing, you want to consider the plane y=y0 (y0=0? Or maybe you’re going to play with it until you get something which looks good) and see what happens to (x,z) as you apply the transform. If there isn’t a perspective transform then you can invert the matrix to get back a mapping from screen coords to world coords and adjust the centre and size of the grid to get something which takes up the right space.

#6

that was my first thought but was not sure :), you render real 3d over faked/pre-rendered 3d, I got it ! ala “Alone in the Dark” :slight_smile:

I started an explanation but as I was not really sure of what you wanted I finally did not posted it (I saved it :slight_smile: )… an image will help a lot to understand

NB: here is an old Real 3D Over Faked 3D Applet Demo I made, the idea was to improve the original idea of Alone in the dark : background are still faked/pre-rendered 3D but using a panorama (6 images) instead of a simple image they are not fixed and camera can rotate.

#7

Yeah, DzzD, that’s exactly what I’m doing. My game is more of a “God” game and less of a FPS-type game, so the camera is simply up in the sky and you can scroll around within, but the scrolling is the draw offset rather than a real perspective change.

So anyway, how did you do this DzzD?

And if I was able to get the transformation matrix from Maya how could I simply throw that into OpenGL?

[EDIT]
Looks like I can probably get the camera’s transformation matrix from Maya, then put it in OpenGL using glLoadMatrixd. The problem is that I really only want to transform the position of the 3d model, I don’t want to change scale or rotation, so using OpenGL to do it may not work. If I’ve gotten Maya’s transformation matrix, how can I apply that transformation to all the 3d models’ positions?
[/EDIT]

#8

If the transformation is a 4x4 matrix, it’s laid out like:


[[     3 x 3    ] [x]
 [ scale and    ] [y]
 [ rotation     ] [z]
 [ 0, 0, 0      ] [1]]

To then take a position in world space, just evaluate T*p where T is the view transformation that Maya gives you. You’re right that with OpenGL, it’d also adjust the rotation and scale of the rendered objects.

The following is conjecture, I haven’t worked out all the math or tested it.
If the 3x3 submatrix is orthogonal (which it should be), so you can transpose it to get it’s inverse. Then when you render an object, you’d use this modelview matrix:

T * Rs^-1 * W

Where T is the view transform, Rs^-1 is the transpose of the upper 3x3 of T (padded to 4x4 using a <0,0,0> translation), and W is the 3d world transform of the rendered object.

This should hopefully cancel out the rotation and scale of the view transform.

#9

[quote]So anyway, how did you do this DzzD?
[/quote]
not at home right now but there are two ways :

1 - perform reverse of translation/rotation to all your object
2 - compute camera local axis in world space (using its rotation & translation) and use it to compute object pos

sol 2 will be cleaner & faster

have to leave now but I will post a full explaination later on the night.

#10

I will admit here that my knowledge of 3D is low and my knowledge of matrix math is barren. As such, I’m probably going to ask some really brainless questions, if you don’t mind. Really the best I’ll be able to understand is something like:

Pt = f(Po)

Where Pt is transformed point and Po is original point, then f is the function that will cause that. If I can get the transform matrix up here could someone maybe baby me through that part? That would be incredibly appreciated.

Also, DzzD, I’m looking forward to your more detailed answer. :slight_smile:

And for anyone who wasn’t quite sure what I’m trying to do, here’s a simple example:

I’m starting with a grid in memory that’s pretty much like this:

http://img522.imageshack.us/img522/6209/normalgrid.png

And I need that grid to end up like this:

http://img518.imageshack.us/img518/6032/picture6xea.png

So you can see that I’m rotating 45 degrees then scaling X and Y dependent upon their position.

#11

All right, I figured out how to get the transformation matrix from the camera in Maya.

I just made a sample camera but here is the result from it:

1 0 0 0 0 0.632401 -0.774641 0 0 0.774641 0.632401 0 0 5.829608 3.798302 1

Which I suppose means:


1	0		0		0
0	0.632401	-0.774641	0
0	0.774641	0.632401	0
0	5.829608	3.798302	1

How do I use all that craziness to to go from:

Po = (50,50);
Pt = (?,?);

Pt = f(Po)

In other words, what’s f()?

#12

To me, the transform is probably this:


1   0    0   0
0   .632401 .77441 5.8296
0   -.774641 .6323401  3.7983
0  0   0  1

Give this a shot:
I’m assuming that (0,0) is the upper left corner in your top-down grid image, and (L - 1, L - 1) is the bottom right.

Let Po = (xg, yg), which is in grid space. Then we can get Pw in world space like this:

 Pw = (sx * (xg - (L - 1) / 2), sy * (yg - (L - 1) / 2), 0)

Where sx and sy are the scales to get the grid into the right size for the camera (it might just be 1 and 1).
pw treats the center of the grid as the origin, and uses a z value of 0 (so it’s a 3d point).

Now, I think all that’s necessary is to call glLoadMatrix() with the above matrix (make sure it’s column major ordering), and then call glTranslatef(pw).

If you’re interested in the final formula, this should be it:
Pw = (x, y, z); // computed as described above
Pt = f(Po) = T * Pw
where T is your view transform:


r00, r01, r02, tx
r10, r11, r12, ty
r20, r21, r22, tz,
0, 0, 0, 1

So, to compute T * Pw, you’d compute:


Pt.x = r00*x + r01*y + r02*z + tx
Pt.y = r10*x + r11*y + r12*z + ty
Pt.z = r20*x + r21*y + r22*z + tz

This Pt should be the point as the camera sees it. To make the shapes render with the correct orientations, you should probably just use the method described above.

I hope that’s right, let me know if it’s what you need/works.

#13

first => evident but important you may first scale your models the same as your world (this will made further more simple), I mean when imported both (background object + object) in the 3d software they must use the same scale.

1 - perform the reverse ops to set your object (for all objects) :

NB : your input object position (px,py,pz) are the pos you get when setting those object at the given pos in maya studio (set it on one case of your grid and get its pos)

for px,py,pz

translate (-tx,-ty,-tz) and rotateZ(-rz) rotateY(-ry) rotateX(-rx)

here you get object pos in 3D camera space (px’,py’,pz’), then you have to project in 2D using same properties as your camera (same fov)

NB : I am not sure about rotation order as I am more working with Max than Maya…

2 - compute camera local axis :

get three vector and an origin (identity matrix):
AX(1,0,0)
AY(0,1,0)
AZ(0,0,1)
O(0,0,0)

apply the same transform to those vectors than the camera :

rotate four vector with : -35.00 -45.00 0.00

and

translate four vectors with : -55.097 50.836 55.111

you then get your Camera local axis and can found 3d pos using their pos in initial 3d world space (get them from maya).

initial 3D position of object in world space is x,y,z

x-=O.x;
y-=O.y;
z-=O.z;
xp=xAX.x+yAX.y+zAX.z; (scalar product P.AX)
yp=xAY.x+yAY.y+zAY.z; (scalar product P.AY)
zp=xAZ.x+yAZ.y+z*AZ.z; (scalar product P.AZ)

3D position of object in camera space is xp,yp,zp

xp,yp,zp are still 3D, so now you have to do projection to 2D (use the same perspective as your cam)

3 - third solution is to use the remap algo I mentionned in first post

this use 2D and only 2D, it use a specific property that linking four corner of a texure (you get it center inded…) but doing the same on the 3d rendered image (using its 2d rendered corner) you also get its center with perspective correct !

see image below

EDIT: added another image sample for third solution

#14

about projection :

double ratio=this.zMin/this.focus;
gl.glFrustum(-ratio/this.zoomX,ratio/this.zoomX,-ratio/this.zoomY,ratio/this.zoomY,this.zMin,(float)Math.min(Float.MAX_VALUE,this.zMax));

with :

focus=1.0/Math.tan(0.5*FOV)
zmin = zmin…
zmax= zmax (not too huge)
use zoomX & zoomY to set your pixels aspect ratio or simply set them == 1.0

#15

This is why I love JGO. :smiley:

Awesome and very detailed responses, guys, thanks. I’ll sort through it all and figure out how to apply it to my test grid that I have right now.

By the way, DzzD, I’m using glOrtho, not glFrustum, because I don’t want the models to change rotation or scale as the camera moves around. I can put glFrustum in if I need to, but anyway I’ll post back later with results.

Eli