Variables assigned in inspector disappear in git version-control?

I'm working on a unity project with a number of people using git version control. When I assign a variable in the inspector (e.g. drag an AudioClip into an AudioSource component), it sometimes disappears when someone else opens the project after I've pushed my changes. This even happens on prefabs when I apply the changes I've made and upload it to the repo.

Does anyone know what causes this problem? Where exactly are the variables one assigned in the inspector saved? Is this maybe a general repo/git problem? Or does it depend on some Unity settings?