Why am I getting random points on a raycast from screen (mouse pointer)?

I'm working on a 3rd-person click-to-move script. I have a simple character controller on a plane and can click-to-move around the plane, but I'm getting some weird behavior when clicking to move behind my character - the point I click jumps around and often will end up nowhere near where the mouse actually is.

Here is my script:

var speed = 6.0;
var gravity = 9.8;
var indicator : Transform;
var stopDistance = 1.0;
var instantTurn = true;
var turnSpeed = 10.0;

private var hasTarget = 0;
private var targetPoint = Vector3.zero;
private var moveDirection = Vector3.zero;
private var grounded : boolean = false;

function Start() {
    if(indicator)
        indicator.gameObject.active = false;
}

function FixedUpdate() {
    if(hasTarget)
    {
        var distanceDiff = Vector3.Distance(Vector3(targetPoint.x, transform.position.y, targetPoint.z), transform.position);
        moveDirection = new Vector3(0,0,speed);
        moveDirection = transform.TransformDirection(moveDirection);
        if(distanceDiff < stopDistance)
        {
            if(indicator)
                indicator.gameObject.active = false;
            hasTarget = 0;
        }
        var targetRotation = Quaternion.LookRotation(Vector3(targetPoint.x, transform.position.y, targetPoint.z) - transform.position);
        if(instantTurn)
            transform.rotation = targetRotation;
        else
            transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, Time.deltaTime * turnSpeed);
    }
    else
        moveDirection = new Vector3(0,0,0);
    // Apply gravity
    moveDirection.y -= gravity;

    // Move the controller
    var controller : CharacterController = GetComponent(CharacterController);
    var flags = controller.Move(moveDirection * Time.deltaTime);
    grounded = (flags & CollisionFlags.CollidedBelow) != 0;
}

function Update() {
    if(Input.GetButton("Fire1")){
        var ray = Camera.main.ScreenPointToRay (Input.mousePosition);
        var hit : RaycastHit;
        if (Physics.Raycast (ray, hit))
        {
            if(hit.collider.gameObject.layer == 8)
            {
                if((hit.point - transform.position).magnitude > stopDistance)
                {
                    setTargetPos(hit.point);
                }
            }
        }
    }   
}

function setTargetPos(target : Vector3){
    //print(target);
    hasTarget = 1;
    targetPoint = target;
    if(indicator)
    {
        indicator.position = targetPoint;
        indicator.gameObject.active = true;
    }
    targetPoint.y = transform.position.y;
}

@script RequireComponent(CharacterController)

I split the setTargetPos function out so that I can set the position when I click on objects around the screen. If I uncomment the print(target) line in that function, I find that my clicks return anywhere from 1 to 4 vectors (depending on how fast I click). In testing it just now, I clicked behind the character controller and it reported two vectors (my plane is transformed 4.2 units up in Y): (1.4, 4.2, 3.2) (-2.2, 4.2, 3.1) If I hold down the mouse button and drag around, I don't get this error.

Edit:

I played with your script for a while and worked out what was wrong

When you do a drag, you're probably (unnoticeably) compensation for the bug, but with a single click (with multiple repeats), what happens is that you start to rotate, and it then starts picking new directions you didn't intend

To fix it, you'll want to hold off on starting the dragging action for 0.25s or so - not enough to be noticeable (especially since you'd still be moving), but enough to fix the bug

Something along the lines of

//variable declares:
private var startTime : float = 0;
private var canDrag = false;
private var firstClick = false;
var singleClickAllowance : float = 0.25;

//in update:
if (Input.GetButtonDown("Fire1"))
{
    startTime = Time.time;
    canDrag = true;
    firstClick = true;
}
else if (Input.GetButtonUp("Fire1"))
{
    canDrag = false;
}

if (firstClick || (canDrag && Time.time > startTime + singleClickAllowance))
{
    firstClick = false;
    //your original raycast code
}