How can I get less clammy on bike rides?

I ride my bike a lot, and often in cold/damp (not really rainy) weather. Even though I wear all the nerdy gear, synthetic fibers and the like, I find myself often having a very hard time regulating my body temperature. Since where I live is very windy, it really makes things worse. The only thing I can think of is switching my sports bra to one that has 0% cotton (mine is currently a mix). Wearing wind resistant layers just seems to make me even more wet. What has worked for you? Also—I am talking about being out on my bike for anywhere from 2–6 hours. Thanks!