How big should I make my prefabs?

In terms of default Physics settings and best-behaviors of floating point values and Raycast() accuracy, your best bet is to treat 1 unit in Unity as 1 meter. It's not a rule (and only barely a suggested guideline), so you don't have to treat it as one, but treating units as meters results in the fewest changes made to keep things "realistic".

That said, that doesn't guarantee that "realistic" will be the most *fun* either, but it's still a good place to start from and work around.

On that note, good luck! It might take a few tries to get the object to be scaled and packaged as you intend in Unity when importing (for example, it might have an additional root object and 100x scale, depending on file format and export settings).