Is the statement "last evening" phrased correctly?

I was talking with a friend and he said that he did something last evening but I told him it is more accurate to say last night or yesterday evening. I know it is proper English to say last night or yesterday evening but is last evening correct?