I have heard a great deal about the concept of sexual dimorphism. It states a subtle external trait that implies male or female in our earliest ancestors would, over time, become exaggerated in a sort of arms race.
@crisw is right, but the effect was likely very slight initially. But in every generation, a high pitched voice was considered feminine and attractive. And in every generation, a deep voice was considered the opposite, strong and manly and virile. They are self re-enforcing mechanisms, making each generation of women have higher voices and men have deeper ones.
Women with long flowing hair are considered sexy. You see its opposite in male pattern baldness.