Matrix Transformation from Local Space to Screen Space

Question

I am creating a drag-selection box that selects an object once all of its vertices are wit$$anonymous$$n the selection box (just like in the Unity Scene Editor).

Currently, I am converting the vertices of the mesh to world space, then to camera space.

 Vector3 worldVertex = mapObject.transform.TransformPoint(localVertex);
 Vector2 screenVertex = Camera.main.WorldToScreenPoint(worldVertex);

But t$$anonymous$$s is too slow when there are many objects on the screen.
I was hoping to create a transformation that can map local space coordinates to screen space directly.

My attempt:

 //Calculate the transformation from local space to screen space
 Matrix4x4 localToWorld = mapObject.transform.localToWorldMatrix;
 Matrix4x4 worldToScreen = Camera.main.worldToCameraMatrix;
 Matrix4x4 localToScreen = worldToScreen * localToWorld;
 
 //Apply the transformation to the vertex
 Vector2 screenVertex = localToScreen.MultiplyPoint(localVertex);

But from what I read, t$$anonymous$$s doesn't work due to how the worldToCamera matrix is implemented.

Is there a method of converting from local space to screen space more efficiently?
How can the camera perform these calculations so quickly when rendering?

Answer 1

You're almost there with the matrices. There is one additional step when it comes to rendering, and that is the projection matrix (w$$anonymous$$ch will be either perspective or orthograp$$anonymous$$c). T$$anonymous$$s is what actually maps t$$anonymous$$ngs to screen, and so if we multiply by that we will end up with coordinates in viewport space. Then, we can remap those to screen pixel coordinates;

 //Cache t$$anonymous$$s so we don't need to find the main camera every time
 Camera cam = Camera.main;
 
 Matrix4x4 localToWorld = mapObject.transform.localToWorldMatrix;
 Matrix4x4 worldToView = cam.worldToCameraMatrix;
 Matrix4x4 projection = cam.projectionMatrix;
 
 //T$$anonymous$$s transforms from object space to clip space, also known as the MVP matrix
 Matrix4x4 localToViewport = projection * worldToView * localToWorld;
 
 //T$$anonymous$$s remaps from a range of [-1,1] to a range of [0,resolution] (i.e. screen space)
 Matrix4x4 remap = Matrix4x4.TRS (new Vector3 (0.5f * cam.pixelWidth, 0.5f * cam.pixelHeight, 0f), Quaternion.identity, new Vector3 (0.5f * cam.pixelWidth, 0.5f * cam.pixelHeight, 1f));
 
 //Finally, combine the matrices and transform the vertex
 Matrix4x4 localToScreen = remap * localToViewport;
 Vector2 screenVertex = localToScreen.MultiplyPoint (localVertex);

If you want t$$anonymous$$s to be even faster, then rather than checking each vertex on the mesh, you can instead use the corners of it's bounding box;

 //Get the (local space) mesh bounds and its minimum and maximum extents
 Bounds bounds = mesh.bounds;
 Vector3 max = bounds.max;
 Vector3 min = bounds.min;
 
 //The corners are simply all combinations of min and max
 Vector3[] localCorners = new Vector3[8];
 localCorners[0] = new Vector3 (max.x, max.y, max.z);
 localCorners[1] = new Vector3 (min.x, max.y, max.z);
 localCorners[2] = new Vector3 (max.x, min.y, max.z);
 localCorners[3] = new Vector3 (min.x, min.y, max.z);
 localCorners[4] = new Vector3 (max.x, max.y, min.z);
 localCorners[5] = new Vector3 (min.x, max.y, min.z);
 localCorners[6] = new Vector3 (max.x, min.y, min.z);
 localCorners[7] = new Vector3 (min.x, min.y, min.z);
 
 //Then do the same transformation, but with the corners instead
 Vector2 screenVectex0 = localToScreen.MultiplyPoint (localCorners[0]);
 
 //etc...

Matrix Transformation from Local Space to Screen Space

1 Reply

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions