Model gets really big when animator controller is applied?

My guess is, you did some resizing of your character after import. Animation doesn't like that. Since animations contain skeleton positions, when you launch it, your bones go to their animated positions (which doesn't take into account your resizing).

If your skeleton animation is included in your import, you can try at-import-scaling. Scaling can also affect physics behavior. So you really should try to model at the right scale from the getgo in your modeling software before exporting to Unity3D, if you want to avoid this kind of problems.