Are the more the merrier?

I often hear this said especially when planning a party or outing. I tend to be fairly selective about whom I want to include in a social gathering and often would rather just be with one other. I was appalled when a friend invited someone on our trip to Florida whom I don’t like. (Later he said that he knew she couldn’t come.) Obviously, there’s no right or wrong answer here but I’m just curious as to what others think.