Can mesh colliders be used for buildings?

Hi, I read some topics about mesh colliders. I want to create a castle with many details (entrance, towers, walls etc.). I used mesh collider. It worked but it's not exactly what I want. It works with upper faces of my mesh. It doesn't colide with walls. Here is screenshot that shows the problem http://www.ifotos.pl/zobacz/Screensho_qnnqhhw.jpg I've attached a simple box collider to the model you can see in the middle of a screen. I set my "convex" in castle collider to unchecked.

User in that topic https://answers.unity.com/questions/1128545/mesh-collider-not-working-for-model.html wrote that "There are some extremely unusual situations where you use mesh colliders, and you in certain situations use them for the "ground", but generally they're never used.".

Is it possibile to do this the way I want? If not, then what should I do? Primitive colliders doesn't work at all with my model.

I'm sorry if such case was described somewhere in the web.