How would you feel if your spouse spent the night at her family's often?

Backstory: my 23 year old boyfriend and his 26 year old sister still live at home together and pay bills. His mom recently got married and moved out of the house to be with her husband whom lives one hour away.

Since its just them two i can see how it can get kinda boring. So she travels down here to spend the night with my bf and his sister 1–2x a week. Sometimes once every other two weeks. Obviously her husband can stay too but doesn’t want to stay with her children lol.

How would you feel if you were the step dad? I think I would be annoyed. Honestly this would make me wonder about my bf and I having our own place together if she spent the night often.