Why am I getting rendering issues, with fog on VR?

Hi,

I am using the Lighting-Tab fog in my scene. The density is set to 1 and i have a second camera, with fog disabled rendering specific objects. With this method I am able to view them in a clean white space without the environment distracting.
This works fine for the normal FPSController, but on my Oculus Quest i am getting rendering issues. When i turn or tilt my head, i can sometimes see the environment. This means, that the fog is not getting rendered right, because only the camera with fog enabled renders the environment. What could be the reason for this?