check for word in dictionary

Question

I'm building a scrabble-like word game and loading a text file containing a list of words that is 80000 words in length.

It seems like it is only loading part of the list and not the whole thing. When I try to access a value past 1000 I get an error. Is there a better way to do this

 var wordCSV : TextAsset; 
 var wordArray = wordCSV.text.Split(";"[0]);
 var wordList = new HashSet.< String >(wordArray);
 
 if (wordList.Contains(wordString)) 
 {
     Debug.Log (wordString + " is a WORD!!");
 }
 else
 {
     Debug.Log (wordString + " not in dictionary");
 }

Answer 1

Best Answer

Answer by Jon Petersen · Apr 09, 2013 at 03:54 AM

I use something similar to the following... I haven't notice any slow down at all and my dictionary is close to 200k words.

 import System.Collections.Generic;
 import System.Linq;
 
 var myDictionary : Dictionary.<String, String>;
 var wordToCheck : String;
 
 function Awake() 
 {
     var MytextAsset = Resources.Load("dictionary", typeof(TextAsset))  as TextAsset;
     myDictionary = MytextAsset.text.Split("\n"[0]).ToDictionary(function(w){return w;}); 
 }
 
 function checkWord(word : String) 
 {
     return myDictionary.ContainsKey(word);      
 }
 
 function Update()
 {
     if(checkWord(wordToCheck))
         Debug.Log(wordToCheck + " is a valid word");
     else if(!checkWord(wordToCheck))
         Debug.Log(wordToCheck + " is NOT a valid word");
 }

Add comment · Show 1 · Share

whydoidoit · Apr 09, 2013 at 03:59 AM 0

Share

Dead right, a Dictionary or HashSet is the right way to go - note that for just lookup then the HashSet uses less memory (and the same algorithm).

Answer 2

Answer by Dracorat · Apr 09, 2013 at 02:32 AM

If your dictionary is sorted (and it should be) you shouldn't be loading the whole thing in to memory. You should access the file and move the file pointer half way in. Check to see if your word comes before or after that point. If before, move the file pointer half way back and check again. If after, move half way after and check again. Keep doing this half and half until your word is found directly in the file. AT MOST - this will require ONLY 16 such searches in a file of 50,000 words to find the right one and will execute blazingly fast with minimal memory footprint.

Add comment · Show 5 · Share

OP_toss · Apr 09, 2013 at 02:36 AM 0

Share

Binary search. I don't think that's faster than a Trie stored in memory. You're still accessing data from disk, which is relatively slow. But I'd be happy to be wrong because that's quite easy to implement! Least you save memory.

Dracorat · Apr 09, 2013 at 02:41 AM 1

Share

A Trie wouldn't be bad either (we answered at the same time). But it requires a lot more memory comparatively.

OP_toss · Apr 09, 2013 at 02:46 AM 2

Share

Agreed. BUT

Binary Search operates at O(log(n)) Trie lookup operates at O(m) where m is the length of the lookup term.

So log(80k) vs like 15 or 20... Much faster. Plus its in memory.

All depends on if memory or speed is more important. And a Trie is harder to implement. You win there.... :P

whydoidoit · Apr 09, 2013 at 03:57 AM 0

Share

200k words which are the absolute reason for the game should definitely all be loaded into memory - I mean, how much can it take 2-4MB only right?

A Hashset or dictionary is an O(1) operation, nothing is faster than that (apart from a better implemented O(1))

maximus202 · Jan 29, 2017 at 07:40 PM 0

Share

I know this is a dead post, but i didn't find anything similar to this on atoher post and i have some question as i am new to c#. I saw the answer but i don't understand it exactly. So what is suppose to initialize the text file with the words in c# and how u compare to see if the word is in it? I tried with dictionar.Contains(string)) is the same as checkWord(wordToCheck))? Also i don't get how i should put the words in text asset? One then this , and another then this, and so on? Thanks for help and sorry if it's any incovenience. @mthicke2

check for word in dictionary

2 Replies

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions