At the North Pole, are east and west just a horizontal circle around the pole?

Since there are no east and west, only southward, at the pole, it makes me wonder where they are. I have a pretty good sense of direction, but having no east or west baffles me. They HAD to go somewhere as we move northward (or southward).

It would seem to me that east and west, which can never be reached, would become circles around the pole as we near it. The sun and the celestial sphere would go in circles around the fulcrum or north point in the sky.