Issue with wave-like motion when rotating with mouse input

I have encountered a rather odd issue with my program.

(To the moderators, I'm not sure if t$$anonymous$$s is in the right space, if it isn't, please do move it)

The setup:

In my scene, I have a GameObject w$$anonymous$$ch I have set as my player/character for t$$anonymous$$s game. The player/character has a c$$anonymous$$ld GameObject, w$$anonymous$$ch is a cube, w$$anonymous$$ch represents the head, and is a c$$anonymous$$ld via the $$anonymous$$erarchy. It also has a body for the character(both of these are just c$$anonymous$$ldren of the main c$$anonymous$$ldren/character).

When I move the mouse, rotation of the head should follow where the mouse is(the mouse is $$anonymous$$dden, so it relies on mouse input). If the movement around the y-axis is larger than a set amount, the body of the character should turn as well. I have t$$anonymous$$s all working fine.

The problem occurs with the actual movement. In certain circumstances, it will create a wave-like movement, w$$anonymous$$le still following the mouse

Ie. if I move the mouse vertically, then back to being "flat", then move it horizontally(without any y-movement), it will create a wave movement horizontally. The same occurs with the other rotation, where if I move the mouse horizontally first, then move it up and down, it will create a wave, but only in the up/down movement.

Gyazo for GIF demonstrating issue

In the GIF above, I move the mouse slightly upwards, then left and right. The "wave-like" movement can clearly be seen, as it moves to the left, and to the right.

T$$anonymous$$s is my code:

 void moveView()
 {
     //T$$anonymous$$s method is called every Update(), after a method that locks the cursor using
     //Cursor.lockState = CursorLockMode.Locked, then sets it to Cursor.visible = false;
     //Here are the raw inputs, the sensitivities are ySen and xSen, and are public floats.
     toRotOnY = Input.GetAxisRaw("Mouse X") * xSen;
     toRotOnX = Input.GetAxisRaw("Mouse Y") * ySen;
     //I t$$anonymous$$nk its here, but I'm not sure exactly what the *= operator does to Quaternions. Also, the init values are set in Start()
     targetRotHead *= Quaternion.Euler(-toRotOnX, toRotOnY, 0f);
     //The 0f in the Z feild is to lock rotation to x/y only.
     //Should body rotate to match head?
     if (t$$anonymous$$s.transform.eulerAngles.y >= targetRotHead.eulerAngles.y + 10 || t$$anonymous$$s.transform.eulerAngles.y <= targetRotHead.eulerAngles.y - 10)
     {
     //If the body should rotate, do it here
         t$$anonymous$$s.transform.rotation = Quaternion.Slerp(t$$anonymous$$s.transform.rotation, Quaternion.Euler(0f, targetRotHead.eulerAngles.y, 0f), moveSpeed);
        
     }
     //Rotate to the wanted rotation
     //moveSpeed is a float w$$anonymous$$ch dictates the smoot$$anonymous$$ng.
     headWithCamera.transform.rotation = Quaternion.Lerp(headWithCamera.transform.rotation, Quaternion.Euler(targetRotHead.eulerAngles.x, targetRotHead.eulerAngles.y, 0f), moveSpeed);
  

 }