You could try relaunching Unity3d. I met with this once, and it kept throwing this error no matter how I modified the code. I relaunched Unity3d and the world regained peace.
Because this was the first hit on google when I had the same issue and couldn’t figure out what I was doing wrong, I had the following offending code:
var bundle = AssetBundle.LoadFromFile(Path.Combine(Application.streamingAssetsPath, "bla"));
var instance = Instantiate(bundle);
When entering play mode that would work, but as soon as I wanted to switch the scene or enter play mode a second time, I got the “Destroying AssetBundle directly is not permitted”
The reason, of course, is just that I instantiated the asset bundle instead of the asset which is inside the bundle, and that’s just crazy The real code for me should look like this
var bundle = AssetBundle.LoadFromFile(Path.Combine(Application.streamingAssetsPath, "bla"));
var asset = bundle.LoadAsset("bla");
var instance = Instantiate(asset);
TLDR;
Check if you are not by accident instantiating a bundle somewhere instead of the asset inside the bundle. If you do, fix the code, don’t save your scenes and restart Unity