What is the difference between pudding in the refridgerated section and pudding in the non-refridged section?

Mostly, it’s going to be like the difference between milk in the refrigerated section, and milk in the shelf-stable boxes. One is preserved (like fruit in a can) and made shelf-stable, one has not been and relies on cold, not sterility, to prevent bacterial spoilage.

The addition of extra chemicals to retard spoilage. compare the labels.

Some of the puddings in the non-refrigerated aisles are dry mixes that require the addition of milk, and therefore are much less prone to spoilage.

The presence or absence of fresh dairy ingredients. UHT dairy doesn’t need to be refrigerated.