What to do about weak WiFi signal?

My old wireless modem died last week * sniff sniff *. I thought my ISP was going to give me a new one of the same model, but no, they gave me a good-for-nothing Chinese piece-of-junk with poor signal strength, ugh! I called tech support and they told me how to increase the strength. It improved things a bit, but some areas of the house still have a so-so signal (places where the signal was just fine with the old modem). Therefore, I’m looking for a device to fix this situation.

Option one is to buy the same modem from some guy on an eBay-type site. It’s relatively cheap, but the warranty is only valid for 2 months, and I have serious doubts about where the modems came from (the guy says they’re in their original boxes from the ISP—WTF? did he buy their leftovers before they switched? did he steal them? I have a bad feeling about the whole thing).

Option two is to buy a router/access point. Here comes the first question: what’s the difference between them? My home network consists of 3 computers and a PS3, all connected wirelessly. What’s best for me?

Do you have any recommendations as to specific devices? I’m looking at this access point and this and this routers.

Whadda ya say?