Do little bits of exercise add up in a meaningful way, or is sustained exercise the only kind that matters to your body?

For example, walking up and down four flights of stairs five times over the course of a day—is that way less useful than walking up and down 20 flights all at once? I have heard exercise doesn’t really ‘count’ unless you sustain a raised heart rate for a long time, but there must be some benefit to very tiny bits of exercise like a few flights of stairs?