Why do animation events have occasional delays?

I am currently working on a fighting game in which our hero can perform various attacks like a punch or a kick.

The damage from these attacks needs to be dealt to enemies on specific frames of the corresponding animation rather than at its start. So I've been using Animation Events to trigger the damage. This is incredibly convenient from a designer's standpoint.

But my problem is that there is an occasional delay between the correct animation frame being shown and the damage actually being received by the enemy. For example I can see the hero's fist being fully extended, and yet the enemy will not take damage until much later. The delay can be up to 250 ms, which is unacceptably long, and yet sometimes it will be 30ms or less, which is perfectly fine and not even noticeable.

So as someone who is fairly new to Unity, my question to you is this: Does anyone know why this delay might be occurring? I haven't seemed to be able to fix it even by deploying a release build. Any pointers much appreciated!