Unusual shadow cast by quad

I discovered a shadow cast on my level when baking lights onto lightmapped surfaces. After disabling a lot of gameobjects I eventually discovered the shadow was a result of a mesh that did not have any faces looking outwards.

I managed to recreate it in a small scene using a green Quad and Red Square

After accidentally baking on CPU I noticed the shadow disappeared. Then I tried on GPU and it appeared again.

Has anyone noticed t$$anonymous$$s before and know of a possible fix without using CPU that takes 6 hours to bake?

Unity 2020.2.5f1 & Geforce 1070