Can I make my computer play Minecraft better?

So, I have a desktop running Windows 7. When I try to play minecraft, it will give me a few minutes of laggy play, and then freeze. I need to open up the task manager and restart the whole application before I can do anything else in it, which may not ruin my save file, but is really annoying. I believe I have it on the lowest quality settings in the minecraft video settings menu, which helps, but does not stop the lag and freezing. After I shut it down, it will say “Java SE Binary is not reponding”, and then go back to normal. Will I need to purchase a better video card to be able to play better? If I do, can you reccomend a fairly nonexpensive one and inclue instructions on installing it?

It seems like most of the questions I ask are about fixing my computers. My parents really need to drop some money on a good computer.