My son and I were bullied on the bus today for speaking English. How should we have handled it?

There was an man on the bus with his own son, (who wouldn’t even sit next to him and sat across the aisle on the window seat), who was speaking very loudly to get his son’s attention on a number of topics and saying what sounded like rather random things very loudly. He wasn’t Norwegian and even made reference to his own language (he said ‘Jambo, Jambo a few times, so I’m guessing his language was Swahili.) as he spoke to nearby passengers in heavily accented Norwegian. As my son and I spoke a few seats behind and on the other side, he over heard us and started commenting to fellow passengers about how we were speaking in English and just what he thought of that. He was very rude about it and very loud about it. I looked up once and glared a bit at him, but otherwise, completely ignored him. A few people had the misfortune of initially sitting next to him when they got on the bus, but, after experiencing his comments, quickly moved away. As my son and I got off the bus, one of the ladies said to him, ‘You should apologise.’ He said very, very loudly and repeatedly, ‘No apology! No apology!’ I just hurried off the bus with my son and ignored the whole thing.