"There is no revenge as sweet as forgiveness." Does this phrase make any sense to you?

Now we all know that we’re supposed to forgive and forget. And everyone tells us that an eye for an eye makes the world go blind. We also know that when we’re extremely angry with somebody we feel like lashing out and doing horrible things to them, and yet it’s common knowledge that forgiving that person will cause all the anger to go away, which is arguably the best feeling one can have.

And yet when I first saw this phrase, something inside me clicked. It somehow makes sense, and yet at the same time didn’t. How can one take revenge upon another by forgiving them? By forgiving them the need for revenge disappears. However, when you forgive someone, isn’t it true that the other person can also feel very bad for what he/she has done? Forgiveness and revenge seem to be mutually exclusive. So is this just one of those stupid phrases that pop up from time to time (shoop da whoop, anyone?)?