Math/Programming Question

I've got the following situation for level editor for the game that I'm building.

http://www.youtube.com/watch?v=PrUuuAOXFuQ

This is the code I have set up for hit.point logic:

void OnGUI()
    {
        if(Input.GetMouseButton(0))
        {
            if(drawing == false)
            {
                oldMouse = Input.mousePosition;
            }
            drawing = true;
        }
        if(drawing == true)
        {
            DrawRect(oldMouse);

            Ray ray = Camera.main.ScreenPointToRay(oldMouse);
            Ray ray2 = Camera.main.ScreenPointToRay(currMouse);

            if(Physics.Raycast(ray, out hit, 200))
            {

            }

            if(Physics.Raycast(ray2, out hit2, 200))
            {

            }
        }
        if(Input.GetMouseButtonUp(0))
        {
            drawing = false;
        }
    }

    void DrawRect(Vector2 pos)
    {
        Rect rect = new Rect(pos.x, Screen.height - pos.y, Input.mousePosition.x - pos.x, (Screen.height - Input.mousePosition.y) - (Screen.height - pos.y));
        GUI.DrawTexture(rect, tex, ScaleMode.StretchToFill, true);
        currMouse = Input.mousePosition;
    }

The camera can be rotated as well. If there was a way to just pelt the entire area with Raycasts that would be cool haha. I've tried a whole bunch of different things, but none of them are very accurate.

http://answers.unity3d.com/questions/27319/accurate-rts-selection

You might be able to use Physics.SphereCastAll, but I think that might end up being too inaccurate - what you could really do with is a CubeCastAll or something! Since that's not available, you might be able to perform a number of Physics.RayCastAll at certain intervals in your selection square to get the accuracy you need.

For example, you have the rectangular area between your oldMouse and currMouse. You could cast a 2D array of rays into the scene, at certain intervals from oldMouse to currMouse. You will probably want to use RayCastAll or SphereCastAll, which will return ALL the objects hit by your cast. So, you might get some duplicates, but you can throw those out and feel pretty good about getting accurate results.

Hope that helps!

the most safe approach is somehow slow. use a capsuleCast and also check the screen space coordinates of objects.

another approach is to check if collider.bound of all objects is inside of a hand crafted bound (using mouse coordinates). it could be inacurate cause sometimes bounds do not match the collider completely.

i think that the rectangle selection can be implemented with any of the methods because %100 acurasy is not really a requirement.

You could try to use Rigidbody.SweepTestAll with a Rigidbody + BoxCollider. The screenspace approach is quite good i think but you can only test for certain points (most likely the pivot).