'Invert' UI mask

I'm trying to fix the same problem. So I come up to the idea to switch stencil operation.

here is a code

 using UnityEngine;
 using UnityEngine.Rendering;
 using UnityEngine.UI;
 
 public class InvertedMaskImage : Image {
     public override Material materialForRendering
     {
         get
         {
             Material result = base.materialForRendering;
             result.SetInt("_StencilComp", (int)CompareFunction.NotEqual);
             return result;
         }
     }
 }

and this is the scene setup

Use the same Mask object on the MaskSprite. but for CoverScreenSprite change Image component to the InvertedMaskImage. so it will replace stencil comparison operation.

Unfortunatly inspector will still show old material parameters, but it will work. in editor as well as in play mode

If you don't want to touch any shaders, there are 3 ways of doing this.

One easy, but the results might be bad (depending on many factors) and one that is somewhat safe, but will require some work on your part to allow it to appears as intended on the screen. I'll explain both ways with, as an example, your idea of a keyhole where the hole is the invisible part of the UI that display the scene or whatever is behind it.

The easy way of doing this is to only use a inverted texture as a UI mask. By using an "huge" UI Image that exceed on all side of the screen, you could have your keyhole set in that.

This is a quick visual reference : The issue with this method is that the hole can't be crystal clear. This means it can work if you're using a smoothed or blurry keyhole right from the start. (By "Resize the Texture", I meant the UI RectTransform of the Image component that hold the keyhole Sprite.)

The other method that doesn't involve making your own or using someone else shader is to do a small trick where you use a much clearer (and possibly smaller) image file of the keyhole inverted. It's like the first option above, but instead of using a HUGE amount of "filler" around the keyhole, you use a set of 3 row and 3 columns where the keyhole is kept in the middle. You can do this by using an intelligent set of Layout groups and uses of Layout Element components. After, you just got to adjust the middle keyhole's RectTransform sizes while letting the 8 surrounding "plain" colored UI element readjust themselves.

The last method requires a bit of work, yet still doesn't need any kind of special shaders. You can use an additional camera with and a higher priority (larger depth) and with Depth Only for its Clear Flag which only see the keyhole texture/mesh (through its culling mask) and animate/move the camera into or out of the the keyhole. This method is a lot more useful if you plan to use some 3D modeled keyhole or if you want more control (via script) over the keyholes animations and/or events. The only issues you might accounter with this last/3rd method is memory/vram wise. Adding a camera for such might be a problem if you're aiming at some light-weight smartphones or tablet as such each camera requires its own Z-Buffering channel to "cut" out the required stuff and put each "results" over each other in the right order.