Many listeners in multiplayer

I am building a multiplayer game with UNet Network Lobby and 3D sound and I am having trouble with the listeners. A listener is attached to each player. After searching I read that I need code that disables the other players' listeners, except the current player's. Not sure how I should do it though - I currently have this script attached to each player, but it only disables the remote client's listener.

  public List<AudioListener> listeners = new List<AudioListener>();
 
  void Start()
     {
  List<AudioListener> listeners = FindObjectsOfType<AudioListener>().ToList();
         for (int i = 0; i < listeners.Count; i++)
         {
             if (listeners[i].gameObject.GetComponent<NetworkIdentity>().isLocalPlayer)
             {
                 Debug.Log("This is my listener");
                 listeners[i].enabled = true;
             }
             else
             {
                 Debug.Log("Listener disabled!");
                 listeners[i].enabled = false;
             }
         }
     }