What should I do about my career?

I have been at my job three months, relatively new to my field. I really like my co-workers, but my one boss treats me very disrespectfully. I am the only one who works with them, which makes it hard to ask for a solution from my office manager (who is also my boss’ spouse, which I wasn’t told when I was hired). I was mislead about a few other things in my job, and it has really rubbed me the wrong way. I have the opportunity to possibly work at another place I would like. I don’t like to give up on things or be flaky, so it bothers me that I might upset a lot of people by leaving a new job so quickly. I also don’t want to be someone who sticks with a job that makes them miserable and doesn’t look at other possibilities. Advice??