Is the term "to go down on" gender-specific?

Please settle this debate between me and an uninformed companion. It would also be helpful if you could cite your approximate age.

EDIT: Approximate geographic location would be helpful as well. Thanks, all.