Removing Inputs from the Input list

I set up a couple custom inputs for one of my joysticks, and I'd like to remove the inputs that are there by default to clean the list up a bit. I've found that just lowering the number of inputs actually removes bottom-up, so that doesn't work. I've been searching around for an answer, and can't seem to find a solution.

Is there any way to remove inputs from the beginning/top of the list rather than the end? Preferably without having to redo the inputs I want to keep.