Door Hinge joint glitches out when pushed to a limit?

Hello! I have a door that is more or less physics-based, you push on it, it opens, normal stuff, and then swings back to the starting position.

But! I am having an issue with if you push the door to the limits (-90 min and 90 max) it starts to spasm on the Y axis and breaks. It can be pushed past the limit too but it will eventually glitch out entirely and snap back into place.

Any fixes for the spasming?