Who created the creator?

This was discussed 5 years ago, but it seems appropriate to ask again as the cast and crew here at Fluther has changed so much in the interveening years, and the question details here are substantially different as well.

Theist often assert that “we know” all things require a cause, and thus the Universe must have had a creator. But lo and behold, the cause they posit just happens to reside completely outside the physical universe, entirely beyond our ability to observe and test, and also just happens to not follow the rule that all things require a cause, the very rule they use to “demonstrate” that this unseen force must exist in the first place.

It’s reminiscent of the woman who claimed at a physics lecture that gravity does not hold the Earth in its place. “What does?” the lecturer asked.

“The Earth sits on the back of a giant turtle.” she declared.

“And what then holds up the turtle?” the lecturer continued.

“It’s turtles all the way down.” she explained.

Isn’t the problem of first cause one of Infinite Regression? Why is a Universe existing in some form eternally, or a Universe from nothing any less logical than an undetectable, supernatural being sitting outside spacetime but responsible for the existence of spacetime and magically controlling spacetime without our ever being able to observe the effects this being produces on spacetime or on cause and effect?