Navmesh agent got fused in very crowded situation when colliding.

I'm trying to make a crowded crossroad simulation where two pedestrian groups crosses the road by following one guide for each group. First what I have done is creating a simple scene using capsule object to as navmeshagent as you can see from my video: [link text][1]

The problem is, if you see at 0:46 the agent fuses and pops out repeatedly on the track. The destination of the agents is their respective leader w$$anonymous$$le the leader itself have a determined destination. The code for the pedestrian can be seen below:

     using UnityEngine;
     using System.Collections;
      
     public class FollowerAgentMovement : MonoBehaviour {
     [HideInInspector]public Transform NPCLeader;
     NavMeshAgent Agent;
     void Start () {
     Agent = GetComponent<NavMeshAgent> ();
     NPCLeader = GameObject.FindWithTag ("LeaderAgent");// each group have different leader, in the implementation there are 2 tag LeaderA and LeaderB
     }
     void Update () {
     Agent.SetDestination (NPCLeader.position);
     }
     }

And t$$anonymous$$s is the code for the guide(leader)

 using UnityEngine;
 using System.Collections;
  
 public class AgentMovement : MonoBehaviour {
     public Vector3 Destination;
     NavMeshAgent Agent;
  
     // Use t$$anonymous$$s for initialization
     void Start () {
         Agent = GetComponent<NavMeshAgent> ();
     }
    
     // Update is called once per frame
     void Update () {
         Agent.SetDestination (Destination);
     }
 }

I'm also pretty aware of the collider and rigidbody effect as stated in http://forum.unity3d.com/threads/need-advice-or-recommendations-with-navmesh-and-agents.130012/

Since I need to detect collision between agent and based on http://docs.unity3d.com/Manual/CollidersOverview.html the collision action matrix of Kinematic Rigidbody Collider cannot trigger the OnCollisionEnter function so I keep using non-kinematic collider as c$$anonymous$$ld of the capsule with rotation and position constrain for the rigidbody to detect the collision and the force didn't affect the navmeshagent component. I also tried removing the rigidbody so it's only the navmesh component that affecting the agent but the fuse problem between agent still exist.

Please someone help me, why t$$anonymous$$s phenomenon happen and how to prevent it? Note: sorry for my bad English. VIDEO LINK [1]: https://youtu.be/G7NVwlpZ544