Question by ExplosiveBarrels · Feb 14, 2017 at 06:34 AM · shader cg fragment

How to get color of a fragment in CG after it's lit?

What I'm trying to do is make my lit meshes look more like sprites by reducing the colors that a given material can draw down to half a dozen or so. For example, a cold grey stone block that normally has a flat blueish gray color for its material, once lit, will be drawn with only 6 or so shades of that blueish gray, varying from black on up to a slightly blueish white, with no dithering.

What I think I need for that is a shader that takes the color of a fragment AFTER it's been lit, and compare it to half a dozen colors specified in the shader, and set the fragment color to the closest matching color from that half dozen.

In order to do that, I need to know what the color is of the fragment after it's been affected by lighting, and that's where I'm stuck. After that, I need to figure out the most optimized way of finding the closest matching color from the ones given.

In combination with a low resolution (384x224) and no anti-aliasing, I'm hoping this reduction in allowed colors per material will make my simple one-color meshes look more like a sprite.

My shader experience is just a step up from absolute beginner. I've written a simple flat color shader, and listened through a few tutorials that explained what's going on inside the CG, but so far had no luck in tracking down how to go about my particular problem.

Also open to alternative approaches, if you think this one is not going to work or is not the best method.

Add comment · Show 2

hexagonius · Feb 14, 2017 at 12:57 PM 1

I'm no shader expert, but as far as I know, that would need a second pass. But I also think, that a better idea is to just write the shader in one go, with light and color adjustment in one go. Looking at this manual page under the section "Calculating Lighting", you should be able to modify it so the final color is calculated along with the necessary light information.

ExplosiveBarrels2 · Feb 14, 2017 at 03:37 PM 0

Thanks! And now this is making a lot more sense, and what I was asking for was pretty simple.

What I need is to take a standard lit shader - the Unity one will do - and add in my comparison and color conversion just before the return statement in the fragment function, since the color the function would've been returning at that point is the lit color I needed.

I'll leave this question open for a bit longer in case anyone has recommendations on how to do the color comparison. My initial thought is to treat the color as a vector and do a dot product on each of the allowed colors, and go with the one that has the highest value from the dot product. But not sure how optimized that is, and this kind of shader will be run on most of the environment meshes in a scene.