Is someone downloading huge files, eating up bandwidth, and making my internet connection sluggish?

I have DSL internet through my local phone company. Occasionally at certain times, usually at night, my internet will largely stop working. Sometimes it simply becomes sluggish, sometimes no sites will load at all, sometimes one or two sites will load, but not others. I have tried everything I know to fix the connection, I have released and renewed my IP address manually, I have used the Windows network connection manager to check the connection, everything says it is fine, except that hardly any packets are being sent and received. I have restarted the computer, unplugged the DSL modem and the router to allow them to reset, all the right lights show on the modem and router. The same problem happens with my machine plugged directly into the router, or with a wireless connection on a laptop or an iPhone. I have turned off my firewall and get the same problem. I assume the problem is the phone company’s, but what exactly is going on? Is there some huge bandwidth hogging activity happening at night? Is there some standard server I can ping to check my connection? Frankly, I hate to call the phone company because they don’t actually give a number specifically for internet customer service and I can’t imagine the parade of idiots I’ll have to go through to actually get someone who can even understand my question, let alone help me. Any thoughts?