How do you remove the plastic taste from new reusable water bottles?

I bought a bunch of reusable water bottles at Wal-Mart which we’re going to fill from our filtered tap instead of buying bottled water. I washed them well in soapy dish water, and filled with water and refrigerated. when i drank two of them today, the plastic chemical taste was so strong. I was thinking maybe vinegar or bleach might do the trick?