Camera shake

I am using the Unity CharacterController Component and it is moving around very well, without shaking. When I parent a camera to it, the camera is following the character and it is not shaking. Now I have created a script that should calculate the camera position (the camera is not parented any more) and follow the character. But now the camera is shaking/jittering. Here is the code from the camera script:

 float distance = 5;
 Transform Target; // Camera should follow this target

 void LateUpdate() {
         rotationX += Input.GetAxis("Horizonzal")* Sensitivity;
         rotationY += Input.GetAxis("Vertical")* Sensitivity;

         Vector3 direction = new Vector3(0, 0, -distance);
         Quaternion rotation = Quaternion.Euler(rotationX, rotationY, 0);
         Vector3 camPosition = Target.position + rotation * direction;
 
         float posX = Mathf.SmoothDamp(mainCamera.position.x, camPosition.x, ref velX, X_Smooth);
         float posY = Mathf.SmoothDamp(mainCamera.position.y, camPosition.y, ref velY, Y_Smooth);
         float posZ = Mathf.SmoothDamp(mainCamera.position.z, camPosition.z, ref velZ, X_Smooth);
         mainCamera.position = new Vector3(posX, posY, posZ);
         mainCamera.LookAt(Target);
 }

PS: The camera is only shaking on a flat terrain, it is not shaking when I walk over other objects or uneven terrain.