What I'm curious about is, we would want to have 100+ options for each part of a costume, can Spine handle this large amount of resources?
Yes, it is possible. Instead of always loading all images, only those used in that skin can be repacked at runtime and organized the texture into only the images needed. See the Runtime Repacking section of the spine-unity documentation for more information:
spine-unity 运行时文档: Combining Skins
We would have about 50 characters with customed costumes on the same scene, same screen. Performance wise, it this going to work well?
To determine that, you need to test it on your target devices. If you have not yet purchased a Spine license, you can download the spine-unity runtime for evaluation purposes and test it using example skeletons included with it:
spine-unity Download
For your information, common measures to improve performance are described in the Spine User Guide and in the runtime documentation:
指标视图 - Spine用户指南
spine-unity 运行时文档: Performance