How can I speed up my game?

Could be any number of things. Since the free version of Unity doesn't come with a profiler, it's hard to know. That's a big reason you should pay attention to your framerate after each and every change/addition you make to your game. Otherwise, you end up in a situation where you've no idea what the problem is.

Some big culprits are usually things like doing too many things every Update, too many physical objects, scene complexity... all obvious stuff. My advice to you would be to do what you've been doing -- removing possible performance degraders and assessing your performance after each removal.

This is based on the assumption that your application is CPU bound, there are many possible factors which can cause a game to run slowly.

The number of lines of code and number of variables is inconsequential. It is often a small section of code which slows things down. This is a good place to apply the '80:20' rule (80% of the time is spent executing 20% of the code). Without a profiler it's difficult to locate which areas of the code are slowest. The easiest method is to remove code and see what difference it makes to the performance.

The easiest way to prevent code from executing would be to rename 'Update' to something else such as 'Update_disabled'. Do this for one class, and run your game and measure the difference. Do that one by one for each class until you find that slow code. If you see a big performance boost after renaming one function, then you have found the problem. Next you can use block comments or preprocessor directives to remove chunks of that function and narrow down the search further.

Once you find the code, optimize it. Generally algorithmic optimizations are better, although code tuning can prove useful in tight loops.