Unity 5: Bloom causes artifacts on Terrain

I recently started polishing one of my levels, and among others I played around with Bloom. I really like what it adds to the visuals, but I have a weird issue, that I've been struggling with for a couple of days. Whenever the terrain is in view white spots appear, and flickers around as I move the camera. When the camera is still, the spots are still as well. These spot vary in intensity, it almost seems like they're like a "shine" or flare or something. I've pointed out a few of the spots on this image:

I am currently using deferred lighting, and if I change to forward rendered, the problem goes away, but I would prefer to be able to use deferred lighting.

• I've tried both gamma and linear color space, with the same results.

• If I remove or disable the bloom image effect (I've tried with both Bloom, and Bloom(Optimized), the problem goes away, but that is not an acceptable solution either.

• If I disable the directional light in my scene (The only light source currently), it get's dark, and the problem also goes away, which have me thinking that this is in fact some kind of shine, rather than a bug.

• Disabling shadows, does not fix the issue either.

At this point I am not really sure what to do.

If anyone out there have any ideas or solutions I'd really appreciate it. I've tried searching for an answer (a lot), but the closest I come is people having issues with terrain trees, and that seems to have been a Unity 5 Beta issue- So if you have an idea, I'd really like to hear it.

Thanks for your time.