Is it just me that whenever I talk to someone and they disregard what I'm saying to them, I take it as a sign of disrespect?

Whenever I talk with anyone, I do my best to give them my attention, in the hopes they will be respectful and listen to me for whatever matter. Now, it seems that I’ve come across some people who for some reason just dont listen to me when I talk to them. They exhibit the behaviors of looking away, not taking much interest in the coversation,etc. It really ticks me off and I get frustrated and only then when I assert that they are being rude, they get angry, criticizing me with more zeal than earlier. It just seems that these people isolate me in their conversations. Is it my paranoia? How do I deal with these people without being so frustrated?