Game feels jittery when moving and looking (first person)

Hi,

I recently swapped to using physics-based movement (applying forces to a Rigidbody in FixedUpdate) with my normal mouselook (Update) and I like it - however, when you move AND look around, there is a jittery sort of effect that happens. If I walk side to side trying to keep my crosshair on a fixed point, the game feels jittery and laggy. I think it may have something to do with using FixedUpdate vs Update but I don't see why and I'm not sure how to sync them up well.

The movement works as follows:

 Vector3 velocity = rb.velocity;
             Vector3 velocityChange = (targetVelocity - velocity);
             velocityChange.x = Mathf.Clamp(velocityChange.x, -maxVelocityChange, maxVelocityChange);
             velocityChange.z = Mathf.Clamp(velocityChange.z, -maxVelocityChange, maxVelocityChange);
             velocityChange.y = 0;
             rb.AddForce(velocityChange, ForceMode.VelocityChange);

And the mouselook works as follows:

             if (axes == RotationAxes.MouseXAndY)
             {
                 float rotationX = transform.localEulerAngles.y + Input.GetAxis("Mouse X") * sensitivityX;
 
                 rotationY += Input.GetAxis("Mouse Y") * sensitivityY;
                 rotationY = Mathf.Clamp(rotationY, minimumY, maximumY);
 
                 transform.localEulerAngles = new Vector3(-rotationY, rotationX, 0);
             }
             else if (axes == RotationAxes.MouseX)
             {
                 transform.Rotate(0, Input.GetAxis("Mouse X") * sensitivityX, 0);
             }
             else
             {
                 rotationY += Input.GetAxis("Mouse Y") * sensitivityY;
                 rotationY = Mathf.Clamp(rotationY, minimumY, maximumY);
 
                 transform.localEulerAngles = new Vector3(-rotationY, transform.localEulerAngles.y, 0);
             }

I tried recording a GIF but the framerate is too low to notice.