Instantiate parts of a prefab/do not instantiate some children of prefab

I have a prefab of my gameobject with some childs for it: sprites, TextMeshPro's and empty objects (I used them as a way to group sprites and text) with their own children. Each child or "group" represents parameters that are given to the prefab during an instantiation. But object dont alway need every parameter to be drawn on it (for example, if a building is free, it doesn't need text telling it's cost and a sprite with a little pile of coins next to it). In that case, they are getting deleted/deactivated right after instantiating, which is obviously bad. What is a best way to avoid creating childs of a prefab to reduce garbage collecting etc?