Why have prejudices & stereotypes got such a bad reputation?

From a survival perspective prejudices & stereotypes are basically essential. They allow us to identify dangers, to quickly categorise people so we can process the information far quicker. However calling someone prejudiced is a bad thing, when did this happen & why? Additionally, people are always saying things like ‘that’s just a stereotype’ as if that means it has no value, why is this? When & why did important survival tools become a bad thing?