Tiny oranges dropping off tree?

I have a potted orange tree (in a huge pot!) that stands 6 or 7 feet high. This is the second spring I’ve owned it. Last year we did not get any oranges. this year it flowered nicely, tiny oranges appeared and when they reached 1/8th or 1/4 inches across they all fell off. Is this normal for a young tree or am I doing something wrong? My lime, lemon and peach tree are grew nice fruit and they get the exact same care and the exact same amount of light.

Any ideas?