What right do we have to deny gay people the right to marry?

I would first like to ask anyone who supports gay rights to PLEASE NOT ANSWER THIS QUESTION IMMEDIATELY. I want to see what kind of response it gets from people arguing against these points before we start too many long winded arguments about who is stupider.

To people who disagree: please think very carefully about you answers and remember that ‘because the bible says so’ is not a valid argument in a society that exercises freedom of religion. Also keep in mind that if you link to a site that has a title like ‘Why Gays are Bad’, you are probably going to have a bad time. Please also consider this and this. While sarcastic, they definitely defuse a lot of anti-gay arguments right off the bat.

Ok, now that the intro is out of the way, I would like to politely ask for ANY reasonable answer to the following:

We have the freedom to life, liberty, and the pursuit of happiness. This started with equal rights for all white men, then blacks, then woman, then disabled people (not necessarily in that order). This includes rights like these and these.
On what basis does anyone in this nation have the right to refuse ANY of these rights to people who have sexual relations with members of the same gender (whether by choice, genetics, or some evil spell the devil cast on them) and would like these protections between them and their chosen partner?