How should I deal with a man who is pursuing me after I've told him not to?"?

Hello! First of all, thanks for taking the time to answer my question. I have a problem… a few months ago I met this guy that started flirting with me but obviously nothing happened, since Im in a relationship and he was in one as well. Recently, he called me and asked me if I could meet him, and I said no. Then I ran into him at a party, and we started talking and basically he said that he wanted to go out with me, but I told him that I couldnt date him because I’m in love with my boyfriend! But still, he called me on Tuesday and then I left it perfectly clear that I love my boyfriend and that I dont want to go out with him, ever. Well, I got a text from him on Wednesday and earlier today! ( In the first one he asked me what I was doing, and on the second one he said that he was in town and that he wanted to see me). I only replied to the first one because since I dont have his number saved, I didnt know who it was. So I just replied ” who is this?” But then with the second message, things started looking a little fishy and I called the number ( from a private line of course, so he doesnt thing that I’m “calling him”) and surprise! His number!!
Now, I dont want to upsted him because I’ve seen this pattern before, with a stalker I had about a year ago. I’m terrified that the whole stalker thing is going to start all over again, and I know that if I say something to upsted this person, he is just going to loose it and will start bothering me. And I think that the answer is to tell him, nicely, to never call me again. But how do I do that? Have you ever been in this position before? Do you think this is a good idea or what should I do? The last thing I need right now is another stalker…
Thank you!