Camera Follow Script destroys Raycast on VR-Hands with Late Update

Hello,

My problem is, that i have a Camera-Follow-Script that follows the player using "Late-Update". It moves my entire VR-Player around, including the VR-Hands on which i have a raycast script with a line-renderer that visualizes the raycast.

The thing is, that the raycast or the linerenderer is flickering up and down. I tried to fix it for a while until i found out, that its actually cause by the late-update in my script that is moving the entire vr-player in which the hands and the raycast are included.

When i change my camera-follow script to "Update" instead of "Late-Update" everything is working fine with the raycast, but the player-follow movement is stuttering.

Here is my Camera-Follow-Script:

public Transform player;

 public float smoothSpeed = 0.125f;
 public Vector3 offset;

 private void LateUpdate()
 {
     Vector3 desiredPosition = player.position + offset;
     Vector3 smoothedPosition = Vector3.Lerp(player.position, desiredPosition, smoothSpeed);
     transform.position = smoothedPosition;
 }

I hope someone can help me. Thanks!