Xbox Controller Input doesn't always go back to 0

When I set up Joystick controls in the Input on inspector for an Xbox One controller it doesn't go back to 0 it stays on the last value anywhere from 0.001 to 1.000 so the play either keeps moving or keeps moving the camera in that direction, This is only updated to 0 when I press a button (Even one that's not set up in the Input), So I came to the conclusion that Unity does not check for Input every frame, At least I think that's why is there a fix for this? I looked everywhere online for 2 days tried using Input.GetAxis("Xbox X") and Input.GetAxisRaw("Xbox X"), Tried changing the gravity, deadzone and sensitivity in all different ways still same issue. I also checked using Xbox Controller on Minecraft and the controller worked fine there, I also tried Game Controller Tester and the axis reset to 0 there. Thanks for the help.