At what point is it considered rude to not have my pan returned?

I have some relatively new neighbors, I’d say they’ve lived there a month or two. Don’t know them very well, but have chatted a couple of times and they seem pretty nice. The other day, one knocked on my door and asked to borrow a baking pan and basically said she’d return it right away.

We’re on day 3 now, still no pan, even though I have seen them come and go multiple times.. even passed them on the stairs yesterday, and no word about the pan.

So I’m wondering at what point is it considered rude they haven’t returned it, and I can knock on the door and get it back?