How natural is it to think of thought as originating inside our heads?

When the ancient Egyptians mummified someone they discarded the brain, which they thought was useless. They believed that the heart was the organ of thought.

When I first heard this it seemed very strange, since the association of head and thought is so ingrained in our culture. It just seems so natural. If we have been concentrating hard on something, we may feel light headed or get a headache. When we get information overload we say we feel as if our head is going to explode.

Is the association natural? Were the Egyptians an anomaly? If not, then when did people realize that our heads are used for thought. Unlike the Egyptians, we associate the heart with love, though we know better, but this makes sense since our pulses quicken when we are feeling strong emotions.