Laggy gameplay and occasional crashes

This problem revealed itself all of a sudden. When we switch scenes from scene A to scene B, the scene B gets really laggy. FPS is around 50, but about every second or so it sinks down to near-zero. Profiler doesn't show anything when these peaks happen since the profiler doesn't keep tracking when this lag occurs, it kinda "skips" the lag part.

I've tried to print out the object counts using both Resources.FindObjectsOfTypeAll() and Object.FindObjectsOfType() functions, and there is no changes in the object / component amounts. I've checked the code and there is little to no arrays in use, and when they are, they are released as expected.

After spending about 30 seconds or so in the laggy scene, I get a GC fatal error saying "too many heap sections". It relates to garbage collection, which in turn relates to memory leaking. It's funny though that if I start the game directly to the laggy scene, it doesn't lag at all, only if I change the scene to it.