How to increase quality of VOIP calls while uploading/downloading?

We have voip phone. When we are uploading/downloading something on the internet (we own a home-based design business, so this happens often), the quality of our calls is horrible (sometimes not even able to use the phone).

What (if anything) would fix this? A higher speed internet connection? Or should we look at switching to a phone line that is not VOIP?

Thanks!