How to Rotate camera around object SMOOTHLY when using Input.GetKeyDown

I've searched on the forum and still cant find how to do this, my problem is that i'm trying to make the camera rotate around the character smoothly with GetKeyDown, but that does not seems to work at all, the camera simply rotate instantly here some code that i have tested:

 private void RotateHorizontaly(bool left)
     {
         float direction = 1;
 
         if (!left)
             direction *= -1;
 
             transform.RotateAround(target.position, Vector3.up, horizontalAngleDegree * direction);
     }

target is the player and this method makes the rotation, and works fine but not smoothly, i used a float direction and a boolean to switch the rotate orientaion when its clokedwised or anticlokedwised, i'm calling this method on a LateUpdate() like this:

  private void LateUpdate()
     {
         FollowTarget();
 
         if (Input.GetKeyDown(KeyCode.Q))
         {
             RotateHorizontaly(true);
         }
         else if (Input.GetKeyDown(KeyCode.E))
         {
             RotateHorizontaly(false);
         }
         
     }

But that does not make the behavior i want, i already tried using Quaternion.Slerp and Lerp but that makes the camera go wild.

The camera is like this:

and i want X rotation and Y position to stay that way, just the Y rotation to move and face the character, can anyone help me?