Advice on how to deal with a small business's terrible customer service?

I’m avoiding some details, since the industry in question is so small, and I don’t want to call them out.
In short, I’ve been dealing with a small one-person business that provides supplies that are hard or expensive to find elsewhere. I’m going to call the shop “Acme” for comedy’s sake.

On September 3rd, I placed an order with Acme, this is my 4th or 5th order with the company, and this was actually for a fairly small quantity of parts. As is the case with this company, they didn’t say anything or ship for at least 3 weeks, when I started inquiring. After some nagging via e-mail, I finally called them, and found out that the parts were out of stock, with an expected in-stock date in 2 weeks.

Mid-October comes, and I drop the proprietor an e-mail inquiring. Turns out the other parts are also out of stock, not expected for another week.

Two weeks later, the order finally ships, the owner nice enough to not charge me shipping. They are two days away via UPS. After the fourth or fifth day, I dropped them an e-mail to inquire what address they had used to ship (I had double checked the order, and my current address is on there, however my last order went to my previous address). Acme had shipped it to my previous address.

We stop by, and find out that it had been returned-to-sender the day before (and a quick stop by the post office they had already sent it back). It was returned to sender this previous Wednesday, and I also haven’t heard anything from the company since.

Any suggestions? I cannot cancel my order and get it from somewhere else, and I’d like to continue doing business with this guy in the future.

Edited to add: The reason this has gone on so long is that I haven’t needed the part until just now. I was trying to avoid lead time hassles by ordering early, and would have shown up right on time had the shipping worked the first time.