Collision not detected between held object and sphere

So I have spent a few hours on this and could not find a solution…

What I am trying to do is pretty simple. The player picks up a golf club, and they should be able to strike a golfball with that club. It is mini-golf, to give you the idea of how much speed would need to be taken into effect.

I cannot find a way to get the user to hit the ball without the club clipping right through, unless the ball is hit very gently. The best way I can think of around this problem is to add extra force when detecting the collision (so that even at low speeds the golf ball will go a fair distance) , but I’m certain that there must be another way to do this since my VR professor had us look at ping-pong demo which has the same concept.

So, how can I do this?

(Also I am using VRTK Interactable Objects for the held object, if that makes any difference.)

You probably want to use continuous collision detection rather than discrete on your rigidbody.