What is the best way to create Stardew Valley-like character customization?

I have started work on a pixel-art RPG game and thought it would be cool to add character customization to it. I've seen Stardew Valley's customization, and I'm wanting to do it that way. My main question is this: do I need to make a spritesheet for every animation, foe every article of clothing, hair style, etc, or is there a better way to do it?

I've looked around, and there don't seem to be any good tutorials or forum posts on the matter.