Reference to components of UI widget

Hey,

after some time developing with Unreal I'm starting to explore Unity. I'm looking for advice for the following task.

In my project the user can click and select objects with the mouse. The object is then highlighted. I'd like to add a widget that displays some information about the object and a context menu with buttons for interaction. The information about the object as well as the callbacks for the different OnClick events are added during runtime.

I've got the widget setup as follows:

 WidgetCanvas
   -TextPanel
      - InfoText
   -ContextMenuPanel
      -ActionButton
         -ActionButtonText (dynamic)
      -DeleteButton
         -DeleteButtonText (static)

So when an object gets selected it should create the widget and provide the information to display and register its callbacks. Here's my question, how do I get a reference to the various UI elements without searching through all child component by name?

My first naive approach was to create an empty gameObject with an attached script and the UI widget as a child. The script holds the references for the UI Elements and provides public methods for registering callbacks, like

 void Start() {
     GetComponentsInChildren<Button>(buttons);
     ActionButton = buttons.Find(b => b.name.Equals("ActionButton"));
 }

 public void RegisterOnActionButtonClicked(UnityAction action)
     {
         ActionButton.onClick.AddListener(action);
     }

This approach feels "dirty" and hard coded. Every time I change the widget or button names I'd have to change my code as well. Is there some common pattern to use?