Is he tired of me?

Me and my friend have been hanging out everyday for a while now. And for the most part, its pretty fun. But we have these moments where I feel I’m starting to annoy him. I’m not a very social person, I don’t really have friends…I’m talking none. And he’s pretty much all I have. Before him, I used to be by myself, did everything by myself, and spent most of my life by myself..I’m used to it. It doesn’t bother me, sometimes I miss it. He always wants to hang out with me, as do I with him, but after a while he kinda just starts to loose interest in me.
My problem is mostly this: He has a lot of friends, and he always wants them to come over or hang out with them…with me tagging along. And its fine, but its not very fun when your the one standing alone in the corner. Its hard for me to socialize even with people I already know. I say a few words, be polite, but I feel as if what I say doesn’t matter so I keep my mouth shut most of the time. I cant help it that I don’t feel very comfortable, but when I mention it to him, it seems like it bothers him. I don’t know what to do, all I want is for him to be happy, but its hard for me too. I tell him he can take me home whenever he’d like, but he doesn’t. And I feel as if I’m ruining his social life. I feel bad, I keep telling him I don’t want to hog him or anything and that he can leave to do his own thing whenever he’d like, but I think he much rather be mad and bored around me. I cant stand it when he gets frustrated with me, I feel like its all my fault and I’m always ready to take the blame and apologize. But its bringing me down, I’m already struggling with things, its starting to get hard handling this too….What should I do?..