Unity's Serialization seems to have a maximum depth of 7. How can I serialize a complex data structure with depths > 7?

I have a Trie (N-tree) data structure with ~100k total elements (large). I need to pre-compute the Trie because it’s very slow (~1.5 min).

I marked it as Serializable, and placed it on a Prefab and pre-computed the data. This worked great! Unity loaded it with very little hiccup and I had all my data in a Trie ready to go!

Then I realized all of my elements of depth 7+ were missing when I used the serialized Trie. It appears Unity Serialization has a max depth cutoff.

So my question is, what is the best way to serialize a complex data structure of high depths in a way that provides very fast load/deserialization time?

My attempted solutions:

  1. Use BinaryFormatter to write my own serialized binary data file to disc, and load it back during the game. This proved horrendously slow and created a huge file (142MB).
  2. Forget serializing and speed up the tree creation by parallelizing it. This still causes a 20-30 second delay on the iPad, vs the unnoticable delay using Unity’s prefab serialization.

Anyone know what Unity is using to serialize and deserialize data? It’s very fast at loading in comparison to anything I’ve tried.

Any help would be greatly appreciated. Thanks!! :smiley:

Thanks for the replies!

Yeah I noticed it seems to be quite popular amongst Unity devs. So thanks for creating it whydoidoit! Why did you do it?!

I’m making a Trie to store a list of words from an english dictionary. Around 100k words. So the tree creation is very slow.

What do you think about the below options?

  1. Read data into a Dictionary/Hashset as they have O(1) lookup and should be faster to create than a complex Trie.
  2. Don’t even read the words in. Instead, whenever you need a match, perform a Binary Search with a file stream, by pointing the stream to halfway through file, and halving back and forth to narrow search results.
  3. Learn and integrate UnitySerializer and manually Serialize and Deserialize my Trie structure to disc as bytes.
  4. Combine 1 and 3 by using Dictionary/Hashset with UnitySerializer.

My main concern, again, is the Deserialization/Loadtime, since I can pre-process the initial Serialization to disc.

Thanks!

EDIT:
First test with HashSet proved very fast at loading/creating! Tests also seem unnoticeable, but I’m testing on a fast workstation… Next step, test on the iPad. Will post results. BTW this cut out like 100s of lines of code (Trie, Node, threading, etc)

EDIT:
Test with HashSet proved quite successful! I got about a 5 second lag hit when building the HashSet from the serialized string. Much faster than the Trie creation! Now I’m thinkin to put that load onto another thread or just add it to the map loading. :smiley: