Bouncing Ball and Walls

I am currently making a version of Pong with a little twist. Everything is going great so far, but if the ball is hit at a vertical/near vertical angle, it will continue to bounce between the two walls and pretty much stays there for the rest of the game. I have primarily made the game using built-in Physics materials, although the paddles and the walls are adding force to the ball when it collides with them(a feature of the game, to make it go faster). The ball has force applied to it at the start. I was wondering whether there would be anyway to prevent this "perfect bounce" scenario, and would be very grateful for any help.

Here are what the walls are doing (the opposite force for the bottom wall):

 function OnCollisionEnter (collision : Collision) {
 if (collision.gameObject == ball) {
         audio.PlayOneShot(WallBoink);
         
         collision.rigidbody.AddForce (0,-20,0);
     }
 }    

And here are what the paddles are doing (opposite force for opposing paddle):

 function OnCollisionEnter (collision : Collision) {
 if (collision.gameObject == ball) {
         audio.PlayOneShot(Boink);
         
         collision.rigidbody.AddForce (-20,0,0);
     }
 }    

Any help would be greatly appreciated.