I feel self conscious and distant from my boyfriend whenever we don't have sex?

We have sex on average 6 times a week but when there is that time when we don’t (whether he or I be tired or stressed, as we are full time University students) I feel like I have no sex drive the next day and that there has to be a reason for us not having sex that has to do with my appearance as well as feeling distant from the relationship. What is wrong? Does anybody else get this?