What is the best way to stay in NY?

I’m staying in New York, NY for 3 days with a lovely young lady at the end of the month. Obviously, it’s very expensive. The nice hostels are all booked for private rooms. My exact questions: is it best to fork up the money to stay at a hotel in the city (better experience, more convenient, etc), or should we just stay in a cheaper hotel in NJ because there isn’t much difference at all? Thanks for any answer and suggestions!