Contollers are unusable when already plugged in when starting build/editor

I've wasted way too much time on this problem so I really hope someone can help! I'm working on a game that uses 2 local controllers, I've been having a problem where sometimes a controller will only update if on that same frame the other controller also updates. The unity input manager is a big blackbox to me so idk what is actually happening, but I've written some pseudo code to show how it roughly seems to work.

  void UpdateControllerInput(int controllerIndex)
  {
        if(CheckIfAnythingChanged(PortListA[controllerIndex]))
        {
             ActuallyUpdateControllerInput(PortListB[controllerIndex]);
        }
  }

Where PortListA and PortListB both point to the input values of the controllers and these lists are normally the same but sometimes they are not. So for example the first and second values of PortListA are flipped in PortListB sometimes.

The inputs don't have to be related in any way, for example if I hold a direction with controller 1 nothing happens, then I press the A button with controller 2 (which is not even used at all in my game) then the direction input of controller 1 will suddenly be read and it will be stuck on that value until I release A (or make the input on controller 2 different in any other arbitrary way) which will again update controller 2.

I've replicated the problem with the newest version of unity (2019.1.0f2) with the following as the only code in the scene:

 using UnityEngine;

 public class ControllerTest : MonoBehaviour
 {
     void Update()
     {
         for (int n = 1; n < 9; n++)
             for (int i = 0; i < 20; i++)
             {
                 if (Input.GetKeyDown("joystick " + n + " button " + i))
                     Debug.Log("Joystick" + n + "Button" + i);
             }
         for (int i = 0; i < 20; i++)
             if (Input.GetKeyDown("joystick button " + i))
                 Debug.Log("Joystick" + "Button" + i);
     }
 }

It will only print an input of the other controller if the other controllers input is also changed that frame. (stick slightly moves, or button is pressed/released)

Now the problem seems to pretty consistently happen if the controllers are already plugged in before starting the editor, and it seems unlikely to happen if you plug them in afterwards. I think there are also some other factors involved for when it does/doesn't happen such as going into sleep mode or playing another game for example, but I'm not entirely sure since it's all very time consuming to test. If it was only the editor I suppose I would be able to live with it (eventhough it would still be very annoying), however it also happens in my builds. I feel like I've tried everything and I really have no idea how to fix it, it seems to me like it's just a problem with Unity, but then it's strange that no one else seems to be having this issue for as far as I can find online. Any help would be appreciated.

Some additional info: I'm using a windows 10 laptop with 2 first party xbox 360 controllers. I never have these problems when playing any other games, so I'm pretty much 100% sure the problem is with Unity somehow.

Edit: I tested it on both my desktop and my laptop and both have the same problem. I can't imagine I'm the only one with this problem but somehow I seem to be.