How can we determine where the north and south poles were in ages past?

There’s the issue with the magnetic pole, and perhaps also the true north which has more to do I think with the stars and Earth’s axis. All of this, I assume, is further complicated by continental drift.

But how could it be determined if a certain geographic location was once the north or south pole, and how long ago?