Gardeners: how close is too close?

I am scared of flying, stinging insects, like bees.

I have recently become the gardener of my own back/side yard. I really enjoy everything about it…almost. I have made peace with bees. Well, it’s a truce, at least.

I’m okay with bees being 4–6 feet away from me just going about there busines.
Today, two bees were lingering around my dwarf Meyer lemon tree and I realized I was abrupt and not careful at all when lifting out dead leaves and such. It’s not that the bees didn’t worry me, I was just too busy to care. One of those things could have easily gotten pissed with me and stung me, but neither of them did.

So I ask, are my docile honey bees ever going to sting me? Do we just learn to coexist? Should I be more careful or just go on merry way, gardening with no fear?