Would you go to your date's apartment if you don't want to sleep with him/her yet?

I just watched The View and the ladies were saying if you don’t want to have sex, you shouldn’t be going to his apartment in the first place. Sex is always assumed if you go to someone’s apartment. If you don’t want to have sex, just end the date before going.

I have problems with this statement… I don’t think apartments equal sex. I’m confused though, since I’m a very introverted person, I don’t have much social intelligence. What do you think?