How do you dump a guy for no good reason?

I know I am going to sound completely shallow for saying this, but here is goes. I’ve been talking to this guy who is really sweet and I know that he really likes me, but he is like three inches shorter than me. I have nothing else against him except for that. He is very sweet, smart (in med school:), athletic (on track team) and has a really hotttt accent but I just can’t get past the height difference. What could I say that wouldn’t hurt his feelings. We aren’t talking talking, just texting every now and then. He’s tried to ask me out before but I always change the subject.