Why do people pressure you to celebrate Christmas?

I don’t celebrate it for religious reasons. Christ wasn’t born then. But anyway, a coworker was very happy about this and asked if I would work for her so she could have off and be with her family and I did. However, then after the Christmas party she asked accusingly “Why weren’t you there?!” I said “Remember, I don’t celebrate it.” Then she said “What is the big deal? It is just a party?” I had one lady tell me off for not celebrating Christ’s birthday but mostly it is just a look of puzzlement when I tell them I don’t celebrate it. I have had relatives buy me and cards and gifts even though I told them I don’t celebrate it and people wish me Merry Christmas knowing full well my beliefs. I still respect those people and all but I am wondering what it is that people just can’t fathom that some others choose not to observe it.