Is it safe to turn off anti-virus/fire walls while online gaming?

Just wondering. I always hate when I get little lag spikes or whatever and plus I figured it would make my online gaming smoother.

Also I use Ad-ware and run a Windows Vista, if that makes any difference.