How do you avoid being a fifth wheel in groups?

This is an issue I keep facing.

I would join a group of people. Sometimes, they just met and sometimes they have known each other for a while. In any case, we become friends or at least acquaintances.

When we meet though for let’s say an informal lunch, they would talk about how much fun they have with each other in events where I was not invited to or present. I used to just ignore that and try to change the subject to something we all like, but then they just continue talking about what they were talking about before.

Then, I told my friends that I would be interested in joining them the next time they do the activity X they all do (e.g. go to the gym together or golfing, etc.). They will either A) say “Well, you probably won’t be interested anyways. You seem quite busy.” or B) say “Sure, we’ll call you the next time we do X.”

More often than not, I don’t get called.

When we meet again, the cycle repeats itself. They talk about fun events that I was not invited to and when I reminded them that I am interested in joining them, they just tell me “We’ll call you. We promise.”

No call. If I confront them about it, which I did once, they say “Oh, I forgot. Sorry.” That’s it. But that was the third or fourth time they have “forgotten.”

So…
1. Should I even consider these people my friends?
2. Is there another way I can get invited to fun events without being clingy?
3. Being a studious person (but not shy), is there a way I can let people know that I know how to have fun too?