When you find love in the least expected place in the least expected person, do you acceptingly follow even though there are consequences?

Small town crushes turn to friendship then to romance with no signs of slowing down. The problem: he has a child and an ex that isn’t very agreeable to the relationship. Having children while I do not-is not the issue. It’s the drama that ensues via his ex. From slashed tires, to slashed reputations, threats and angry outbursts towards me. Many feel I should walk away, but I feel I should stay for the love we’ve found in eachother and strength and trust is unparallel to anything we’ve found before. Also, we have become better people with eachother and have grown so much in the short time we’ve been with eachother. So handle the consequences? And if so how? Or walk away?