Calculation behind camera.WorldToScreenPoint

Question

Hello all,

Is there any way to map world point to screen point using matrix, which output value same as one get when using Camera.main.WorldToScreenPoint function?

Answer 1

Best Answer

Answer by zach-r-d · Jul 25, 2015 at 12:30 PM

Unfortunately, this is mathematically impossible. One of the operations involved in computing the screen point of a world coordinate (dehomogenizing a homogeneous coordinate) cannot be represented as a linear transformation, so there cannot exist a matrix (which is really just a convenient encoding of a linear map) that can represent it.

If you're curious how the function actually works, it probably looks something like this:

 Vector3 manualWorldToScreenPoint(Vector3 wp) {
         // calculate view-projection matrix
         Matrix4x4 mat = cam.projectionMatrix * cam.worldToCameraMatrix;
 
         // multiply world point by VP matrix
         Vector4 temp = mat * new Vector4(wp.x, wp.y, wp.z, 1f);
 
         if (temp.w == 0f) {
             // point is exactly on camera focus point, screen point is undefined
             // unity handles this by returning 0,0,0
             return Vector3.zero;
         } else {
             // convert x and y from clip space to window coordinates
             temp.x = (temp.x/temp.w + 1f)*.5f * cam.pixelWidth;
             temp.y = (temp.y/temp.w + 1f)*.5f * cam.pixelHeight;
             return new Vector3(temp.x, temp.y, wp.z);
         }
     }

Add comment · Show 7 · Share

amit_nath30 · Jul 25, 2015 at 12:36 PM 1

Share

This is exactly what i was looking for. thank you

zach-r-d · Jul 25, 2015 at 01:42 PM 0

Share

Glad it helped! Remember to mark an answer as accepted if it answers your question. :)

Phylliida · May 07, 2017 at 10:21 PM 0

Share

How do you invert this? (i.e., what is manualScreenToWorldPoint)

JanSenseVR · Nov 13, 2019 at 12:41 PM 0

Share

Inverted function - ManualScreenToWorldPoint

To get the required matricies you can pass in camera.cameraToWorldMatrix and camera.projectionMatrix.inverse.

Hope this helps someone someday :)

   public static Vector3 ManualScreenToWorldPoint(Vector2 screenPoint, float distance, Matrix4x4 cameraToWorldMatrix, Matrix4x4 projectionMatrixInverse) {
     // here we are converting screen point in screen space to camera space point placed on a plane "distance" away from the camera
     // screen point is in range [(0,0) - (Screen.Width, Screen.Height)]
     Vector2 pointViewportSpace = screenPoint / new Vector2(Screen.width, Screen.height); // convert space [(0,0) - (Screen.Width, Screen.Height)] to [(0,0) - (1,1)]
     Vector2 pointCameraSpaceNormalized = (pointViewportSpace * 2.0f) - Vector2.one; // convert space [(0,0) - (1,1)] to [(-1,-1) - (1,1)]
     Vector2 pointCameraSpace = pointCameraSpaceNormalized * distance; // convert space [(-1,-1) - (1,1)] to [(-dist,-dist) - (dist, dist)]
     Vector4 planePoint = new Vector4(pointCameraSpace.x, pointCameraSpace.y, distance, distance); // define the point (don't know why z and w components need to be set to distance)
 
     // calculate convertion matrix from camera space to world space
     Matrix4x4 matrix = cameraToWorldMatrix * projectionMatrixInverse;
     // multiply world point by VP matrix
     Vector4 worldPoint = matrix * planePoint;
 
     return worldPoint;
   }

JanSenseVR · Nov 13, 2019 at 12:53 PM 0

Share

The convertion other way around could look something like this. Tested and works without issues.

ManualScreenPointToWorldPoint function

   public static Vector3 ManualScreenPointToWorldPoint(Vector2 screenPoint, float distance, Matrix4x4 cameraToWorldMatrix, Matrix4x4 projectionMatrixInverse) {
     // here we are converting screen point in screen space to camera space point placed on a plane "distance" away from the camera
     // screen point is in range [(0,0) - (Screen.Width, Screen.Height)]
     Vector2 pointViewportSpace = screenPoint / new Vector2(Screen.width, Screen.height); // convert space [(0,0) - (Screen.Width, Screen.Height)] to [(0,0) - (1,1)]
     Vector2 pointCameraSpaceNormalized = (pointViewportSpace * 2.0f) - Vector2.one; // convert space [(0,0) - (1,1)] to [(-1,-1) - (1,1)]
     Vector2 pointCameraSpace = pointCameraSpaceNormalized * distance; // convert space [(-1,-1) - (1,1)] to [(-dist,-dist) - (dist, dist)]
     Vector4 planePoint = new Vector4(pointCameraSpace.x, pointCameraSpace.y, distance, distance); // define the point (don't know why z and w components need to be set to distance)
 
     // calculate convertion matrix from camera space to world space
     Matrix4x4 matrix = cameraToWorldMatrix * projectionMatrixInverse;
     // multiply world point by VP matrix
     Vector4 worldPoint = matrix * planePoint;
 
     return worldPoint;
   }

Example usage (place on the camera gameobject and play around with screen point and distance fields)

 using UnityEngine;
 
 public class DrawManualScreenPointToWorldPoint : MonoBehaviour {
   Camera cam;
 
   [SerializeField]
   float distance = 1000;
   [SerializeField]
   Vector2 screenPoint = Vector2.zero;
 
   void Start() {
     cam = GetComponent<Camera>();
   }
 
   void Update() {
     var point = MyUtilityClass.ManualScreenPointToWorldPoint(
       screenPoint,
       distance,
       cam.cameraToWorldMatrix,
       cam.projectionMatrix.inverse);
 
     Debug.DrawLine(cam.transform.position, point, Color.green);
   }
 }

Show more comments

Calculation behind camera.WorldToScreenPoint

1 Reply

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions