How much does a person's personal life matter when running for political office? Would it change your vote if it was a negative situation?

Okay, I was watching The View this morning, and they were talking about how personal lives should or should not be factors in running for president. They mentioned Newt Gingrich, who said this:

“There’s no question at times of my life, partially driven by how passionately I felt about this country, that I worked far too hard and things happened in my life that were not appropriate,” he said during an interview with CBN..

He’s not running for president at this time, but I was thinking. Would his personal life have anything to do with your vote? I mean, with his three marriages, etc.