Where is it established that some groups may be identified by their skin color and others may not?

In the West, it is pretty much accepted that people of sub Saharan ancestry who have dark skin are called “Black”
Just like people of European ancestry are called “White”

I can’t prove it, but I suspect that anybody in the Western world who called indigenous Americans “Reds”, or called people native to the Far East “Yellows” would probably get fired, be forced to resign, or have to withdraw from the political campaign.

Who makes those rules?