Unity 5 Screen Overlay Image Effect banding issues

When my team switched to Unity 5 the screen overlay effect looked a lot worse and didn’t act like how the overlay blending effect works in other programs.

It’s easiest to explain with images:

no camera effect / Unity 4 screen overlay / unity 5 screen overlay

You can see the Unity 5 overlay has this weird banding in brighter areas. I tested this on a default project and got the same result.

I was wondering if anyone has seen similar issues and if there’s a way to fix it?

TL;DR: changing line 69 in the shader from step(0.5, color.rgb) to step(half3(0.5), color.rgb) will make it work again.

In 5.0 we’ve changed the shader compiler for d3d9. It looks like in this case it behaves differently (incorrectly) and doesn’t promote the 0.5 scalar to a vector anymore. Instead it picks a channel from the second argument, so the threshold check is no longer valid for the other two channels and the overlay curve gets a discontinuity (the band).

We’ll fix it in Unity as well, of course.

BTW, even though the shader for the overlay mode looks a bit strange, it’s the “canonical” overlay function. My guess is that someone figured out it’s fewer alu ops when written this way, but failed to add a comment or clean it up.