How to toggle elements within a prefab in Unity

I'm still quite new to both scripting and Unity so I apologize if my question is a little vaguer than it should be. If clarification is needed I will do my best to clarify.

I'm currently working on creating a map generator in Unity using C# and have hit a bit of a snag in terms of how best to approach something. I've currently got Prefabs with the names of each possible co-ordinate in a grid (A 5x5 square, A1-5 going to E1-5) that contain every possible tile as a gameobject. Separately I've also written a simple shuffler based on the Fisher-Yates method to shuffle a list of integers with values representing each possible tile and then they are assigned to integers used to represent each location on the grid (e.g. int A1 = 15). What I am trying to accomplish could be visualized as a deck of cards being shuffled and then one card is placed on each section of a grid to build a level.

However, the main issue I'm having is the following: I currently have a prefab with tiles labelled 1 through 17 (each possible tile that could be on that space in the grid) but I want to de-render all the tiles that are not currently needed within the prefab. For example, if A1 = 15 then I want tile 15 to be displayed, as well as its hitboxes, and I want the rest to be disabled so that they cannot be seen or interacted with. Is there a way to simply have a prefab spawned to correspond with the integer? If not, how would I go about writing a C# script having individual elements of a prefab despawn/de-render while keeping the ones that I need? I'll also be needing to update the game as it goes on so I'll be needing to keep this method somewhat dynamic in order to be able to update sections of the grid when needed.

Thank you again, sorry if I haven't been fully clear.