Disable physics reaction

In my game I have object with non-kinematic rigidbody which is moved using transform.Translate() method (guess this way I'm not getting CCD even if it's turned on?). And I have OnCollisionEnter(Collision col) where I react on collision, using contact points and other info from collision.

And all seemed ok, but then I made it possible to pause the game. When paused - I do not update the position of my objects, so they should stay still. But then I clearly saw my object slowly moving in the direction of his velocity (or similar direction). My guess is that somehow object's rigidbody still reacts with other rigidbodies on collision thus gaining itself some acceleration. That's not what I want. Also, when I tried spamming objects and watched them colliding with each other at some point they began to jump off each other with tremendous speeds. Which is probably due to physics reaction too.

So, is there a way to disable the reaction, yet still get the collision info (like contact points, normals and such)?