Can you help me understand audio formats (please see inside)

I have audio files (such as music) on my Windows 7 PC that are mostly MP3, but some are AAC and some are WAV. I can also download audio or the audio portion of a video in any of the above formats as well as M4A, WMA, and OGG.

My CD player cannot play MP3s. What format should the audio file be, so that I can play it on my CD player? (I’m talking about a separate CD player, not the one in my PC.)

Can I convert the MP3s to that format, and what software is best for this purpose? If I don’t want to install new software, is there a website that I can use for the conversion?