FindObjectsOfType vs List.Contains Performance?

There is something exactly for this, a dictionary. It can use the TryGetValue call which is a very fast call to retrieve / cache already found objects by a unique key. Basically you give a key and a value to a dictionary, where you can then retrieve the value by its key. Here's an example of pseudo code where I use an int as key and gameobject as value:

 Dictionary<int, GameObject> cachedObjects = new Dictionary<int, GameObject>();
 
 GameObject connectedObject; //our holder for when we do find the object
 // Locate the other gameobject by specific id
 if (cachedObjects.TryGetValue(id, out connectedObject)) {
 // If we already have some object by this specific ID then do w/e
 // our gameobject will be inside the connectedObject variable
 }
 else 
 {
 // Do w/e expensive call to retrieve the object you require
 //add the retrieved object by the specific id as key so we can get it fast nexttime
 //without having to do the expensive call to retrieve it again
 cachedObjects.Add(id, retrievedgameobject); 
 }

Do note a dictionary can only have UNIQUE keys. If one key can store more objects then your value will have to be a list of gameobjects like so:

 Dictionary<int, List<GameObject>> cachedObjects = new Dictionary<int, List<GameObject>>();

 List<GameObject> connectedObjects;
 if (cachedObjects.TryGetValue(id, out connectedObjects)) {
     // we already have a list of objects so we can just add to it or use them
     connectObjects.Add(someOtherObject);
 }
 else 
 {
 // we don't have a list yet
 var listdata = new List<GameObject>();
 listdata.Add(someitem);
 // do some expensive calls and add items to the list, then add the list with key to dictionary
 cachedObjects.Add(id, listdata); 
 }