Get the color of a WebCamTexture at the click position

I need to pick the color of a WebCamTexture with a mouse click at this position and render the color to a cube.
My scene consists of a quad with the script below and a cube. The quad shows the video content. The cube should get the color of the position where the quad was clicked.

However the cube always gets the color of lower left pixel of the WebCamTexture, no matter at which position of the quad I click to. The RaycastHit seems to output the right coordinates, but it seems to me the GetPixel method doesn´t use the right coordinates.
I have seen multiple forum posts but none of them helps me further.

Can you please lift me up?
Thanks in advance.

using UnityEngine;
using System.Collections;

public class webcam : MonoBehaviour {
	public WebCamTexture wct;
	public GameObject cube;

	void Start () {
		wct = new WebCamTexture ();
		GetComponent<Renderer> ().material.mainTexture = wct;
		wct.Play ();
	}

	void Update () {
		if (Input.GetMouseButtonDown(0)) {
			RaycastHit hit = new RaycastHit ();
			Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
			if (Physics.Raycast(ray, out hit)){
				Debug.Log ("Click position: " + hit.point);
	
				Color color = wct.GetPixel ((int)hit.textureCoord.x, (int)hit.textureCoord.y);
				Debug.Log ("Color x/y: " + color);
				cube.GetComponent<Renderer> ().material.color = color;
			}
		}
	}
}

Reason of your error: The raycast hit point represents the intersection point in world coordinates.

I’m assuming your quad texture fills the entire screen

  1. Your world coordinates have to be converted to screen coordinates. You can use Camera.main.WorldToScreenPoint method to get your screen coordinate.

  2. You need to convert the screen coordinates to a ratio. The screen point can’t be used as value for the texture coordinates because screen resolutions may be different from the camera texture size. Divide both X and Y by Screen Width and Screen Height to get a 0-1 ratio:

  3. Sample texture based using the ratio. You can get the correct pixel by multiplying the texture size by the ratio.

  4. Get the pixel using the coordinate.

PS: The reason you’re getting the last pixel is because your texture wrap mode is set to clamp, but changing it to repeat wouldn’t fix your issue, It would just give you a different result.

PS2: If your camera texture doesn’t fill the entire screen (UI element such as RawImage), instead of getting the ratio based on the Screen size, you would have to calculate the ratio based on the UI rect. The formula could vary based on the UI alignment and anchoring.

Thanks for your detailed answer.
I tried to get through it. Step 1 works fine.
At step 2 I don´t know exactly what to do.

  1. You need to convert the screen
    coordinates to a ratio.

I added a Unity Package so that you can see what I’m doing.
After step 1 I could render colors other than the lower left pixel´s color, but they were not the right colors. After step 2 the result was worse, so I commented it out (script below and in the Unity Package).
By the way, my texture does not fill the whole screen.
I think it might be easiest for you, me and the other readers on this thread to work with a sample scene, don´t you think so?
Thanks in advance.

EDIT:

I tried the same with a movie (another Unity Package attached). Unfortunately, it´s the same effect: I do not get the right color. Someone should explain step 2 to me.

using UnityEngine;
using System.Collections;

public class colorPicker : MonoBehaviour {
	public WebCamTexture wct;
	public GameObject cube;
	Camera cam;

	void Start () {
		cam = Camera.main;
		wct = new WebCamTexture ();
		GetComponent<Renderer> ().material.mainTexture = wct;
		wct.Play ();

	}

	void Update () {
		if (Input.GetMouseButtonDown(0)) {
			RaycastHit hit = new RaycastHit ();

			Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
			if (Physics.Raycast(ray, out hit)){
				Debug.Log ("Click position: " + hit.point);
				Vector3 hitpoint = hit.point;
				Vector3 screenPos = cam.WorldToScreenPoint(hitpoint);


				Color color = wct.GetPixel ((int)screenPos.x, (int)screenPos.y);
				// Color color = wct.GetPixel ((int)(screenPos.x/Screen.width), (int)(screenPos.x/Screen.height));
				Debug.Log ("Color x/y: " + color);

				cube.GetComponent<Renderer> ().material.color = color;
			}
		}
	}
}