Why can't the Olympics be in the same place every year?

I know each country wants to host the games, but we build absurdly massive stadiums ever two years that see about two weeks worth of use. Its as if multi-billion dollar stadiums are disposable. Yeah its unfair to give the same country the Olympics each year, but I think thats something we need to get over (or give the games to Antartica or the Vatican?)

How about we build a super awesome summer olympic stadium and a mega awesome winter stadium in two different countries and use them for every Olympics?(maybe we could give them back to Greece?) That way we wouldn’t have to displace tons of civilians every time a new city gets it, nor would there be all the environmental damage nor the financial loss from building a new one every two years.