Extend the Collider When Picking Up Objects

So I have looked around and so far I haven't been able to find a good answer to this question. In my game I have my player so he can pick up objects (like in Portal, Half Life Two, Amnesia etc.) however I have run into a problem when the object the player has picked up collides with something that doesn't move (like the wall, the ceiling, the floor, various props and so on).

 #pragma strict
 
 var hitObject : GameObject;
 var isColliding : int;
 isColliding = 0;
 
 function Update()
 {
     if(Input.GetMouseButtonDown(0)){
 
         var hit : RaycastHit;
         // Cast a ray
          
          /************Dragging************/
         /************Dragging************/
         if (Physics.Raycast(Camera.main.ScreenPointToRay(Input.mousePosition), hit, 3) && hit.collider.tag == "Pickup")
         {//is it a pickup type
             hitObject = hit.collider.gameObject;
             hitObject.rigidbody.isKinematic = true;
             hitObject.transform.parent = gameObject.transform;
         }
         /*******************inventory stuff************************/
         /*******************inventory stuff************************/
     }
     if(hitObject != null && (Input.GetMouseButtonUp(0) || !Physics.Raycast(Camera.main.ScreenPointToRay(Input.mousePosition), hit, 3))) // This will release the object 
     {
         hitObject.transform.parent = null;
         hitObject.rigidbody.isKinematic = false;
         hitObject = null;
     }
     /***********more inventory stuff***********/
     /***********more inventory stuff***********/
 }

The way I currently have it once click on an object will either be dragged, picked up, or placed in the inventory. The dragging and placing in the inventory works well. Because they do not affect the pickup up of the object I have left them out of the script to make it easier to read. What I would like to have happen is for the collider to still function and keep the object from going through the objects so easily. However I do not want the movement of the object to move the player (rotating the object into a wall on the left does not moves the player to the right).

Thanks a million!