How can I help my lemon tree to grow lemons?

I have a small lemon tree in my backyard in San Francisco. I bought it last year and planted it in a large container (it’s the kind of small meyer lemon tree that I’ve seen growing in containers). I was excited to see that it has lots of flowers this year, but so far none of them seem to be turning in to lemons. The backyard does not get a lot of light since it has buildings on all sides, but I put the container with the lemon tree in the sunniest part of the yard.

Is there anything else I can do to make my lemon blossoms turn into lemons? Pruning? Fertilizer? Or is it just not going to happen because the tree doesn’t get enough direct sunlight?