Custom shader that behaves like Sprite mask

I want to create a custom shader that behaves like the SpriteMask component, where I could add some extra conditions.

I tried to create a material with my own copy of the “Sprites/Mask” shader, and apply it to a quad, but that didn’t work, so clearly there’s more going on here.

What you want to do is make a shader that uses the stencil buffer. You can read about it in the docs here: https://docs.unity3d.com/Manual/SL-Stencil.html