Why is misjudging time worse than misjudging space?

I am both chronically late (misjudge time) and pathologically clumsy (misjudge space). Why do we as a society think it’s rude to be late to stuff but falling off your bike or spilling your coffee is completely different? (Especially if it’s something like a lecture or a class and you’re not inconveniencing anyone other than yourself by missing the first five minutes?)