Follow mouse x position and move forward

I have a Game Object that I want to be on constant forward movement and based on mouse x position to move sideways. But I am struggling to combine both movements. In the below code if I comment line number 2 you side movement is OK otherwise is not working properly.

     void Update() {
         player.transform.position += transform.forward * 30 * Time.smoothDeltaTime;
         //player.transform.Translate(0, 0, 1 * 30 * Time.smoothDeltaTime);

         if (Input.GetMouseButtonDown(0)) {...}

         else if (Input.GetMouseButton(0)) {
             Ray mRay = Camera.main.ScreenPointToRay(Input.mousePosition);
             float rayDistance;
 
             if (raycastPlane.Raycast(mRay, out rayDistance)) {
                 player.transform.position = new Vector3(mRay.GetPoint(rayDistance).x, player.transform.position.y, player.transform.position.z);
             }
         }
         else if (Input.GetMouseButtonUp(0)) { ...}
     }
 
    Ray GenerateMoseRay() {...}