What to do about neighbours who are very disrespectful of my son...

Well, Spring has sprung and the sun is up longer and the kids are playing outside again. My neighbours have two boys and they pick on my son but my son tries and tries to keep engaging them in play. Sometimes it works out just fine, but most of the time it ends with someone in tears and stomping their feet home. They have a trampoline and have forbidden my son from playing on it. Just now, the mother came out and screamed at my son to get away from the trampoline because he was trying to play with the boys who were jumping on it. I heard this all because I was already there, trying to get my son away from it because I know what her reactions are like. I allowed her older son to park in my drive way a few times when he asked, but now he does it all the time without even asking and he hasn’t even offered to do anything for it, like cut my grass. Grrrr. Is it time for me to get tough with my neighbours?