Restrict held object movement

I allow my player to pick up objects and have them follow the cursor, which presents 3 unique problems:

1. when i hold the object, i set it's transform.position to the Camera.main.ScreenPointToRay, however, it phases through the walls of my building (both have mesh collider, rigidbody)... The building has kinematic enabled on the rigidbody, and the objects, when not held have it disabled so they can sit on the building floor. When i pick up the object i turn kinematic on the object, and gravity off so it sticks to the screen point. However, when i do this it won't be affected by the wall since they are both kinematic.

2. some assets don't have the proper rotation. however, if i try to parent it to a corrected object, then the raycast to pick up the object won't find the collider of the child

3. i have weights for the objects, obviously a 1lb object shouldnt knock around a 1000lb object, but i'm not sure how to make this happen. Rigidbody.mass doesn't help