Obtaining a depth Point Cloud

Hi,

I've been trying to extract a point cloud from Unity w$$anonymous$$ch has depth information for a w$$anonymous$$le now, but have had some trouble. I found some links online to help me get started and I feel as if I'm almost there, but there is slight problem - what should be straight lines appear to be curved. I was hoping someone could help me locate the issue / point me in the right direction.

To start off, to obtain the depth information, I used t$$anonymous$$s approach: http://answers.unity3d.com/questions/877170/render-scene-depth-to-a-texture.html

TLDR for the link: The shader and the code produce a greyscale image w$$anonymous$$ch represents the depth in the scene, where black is closest to the camera, and w$$anonymous$$te is the farthest. The code and shader used is the same as the ones in the reply of the user who posted t$$anonymous$$s.

Running the code produces t$$anonymous$$s image in Unity w$$anonymous$$ch looks proper:

From there, I obtain the render texture in the same way as the link above and store the pixel information in an array. Here is a quick reference:

         int resolutionX = Screen.width;
         int resolutionY = Screen.height;
 
         RenderTexture tempRt = new RenderTexture(resolutionX, resolutionY, 0);
         camera.targetTexture = tempRt;
         camera.Render();
         RenderTexture.active = tempRt;
         Texture2D tex2d = new Texture2D(resolutionX, resolutionY, TextureFormat.ARGB32, false);
         tex2d.ReadPixels(new Rect(0, 0, resolutionX, resolutionY), 0, 0);
 
         string[] output = new string[resolutionX * resolutionY];
         Color[] pixelInfo = tex2d.GetPixels();

From here, I run the following code on each pixel to attempt to translate it into real x, y, and z coordinates and output it into a file w$$anonymous$$ch is to be my point cloud.

             float r, g, b;
             double x, y, z, fl, range;
             double realX, realY, realZ;
 
             // Obtain X and Y Pixel Coordinates
             double pixelX = i % resolutionX;
             double pixelY = i / resolutionX;
 
             range = pixelInfo[i].r;
             x = (pixelX / resolutionX) - 0.5;
             y = (-(pixelY - resolutionY) / resolutionY) - 0.5;
 
             fl = -0.5 / (Math.Tan((Fov / 2) * Math.PI / 180));
             z = -fl;
 
             double vecLength = Math.Sqrt((x * x) + (y * y) + (z * z));
 
             // r = g = b because we are getting the value from the depth grayscale image
             r = (int)(pixelInfo[i].r * 255);
             g = (int)(pixelInfo[i].g * 255);
             b = (int)(pixelInfo[i].b * 255);
 
             // unitize the vector
             x /= vecLength;
             y /= vecLength;
             z /= vecLength;
 
             // multiply vector components by range to obtain real x, y, z
             realX = x * range;
             realY = y * range * -1;
             realZ = z * range;

When reading the file into Meshlab, all looks more or less okay, EXCEPT my straight lines are not as straight as they should be.

Image of the scene in Meshlab, rotated to show curvature.

As you can see, the rectangle w$$anonymous$$ch is a cube in the Unity world is curved when it should be straight and I have no idea why. Any help is appreciated. If anyt$$anonymous$$ng is unclear I'll gladly clarify. Thanks!

EDIT: I got the answer by posting in the forums. All I had to do was normalize the z component of vector to 1.