Why do clothes get that musty smell in the closet?

I store winter clothes in a different closet during spring and summer and switch out clothes in the fall. The clothes in the “storage closet” get this nasty smell and I always have to wash them before I wear them. The clothes in my “in use” closet never get that smell, even if I don’t wear them regularly. Is it because the door gets opened and shut regularly on the “in use” closet but not on the storage closet? What makes the smell anyway?