Is there any way to remove excess grout and save my mosaic tiles?

My friends and I did a mosaic tile project on paving stones for my backyard. We glued the tiles on and then bought grout at an arts and crafts store. We spread grout over the tiles and followed the instructions to let it dry before using a scrubby sponge to remove the excess grout from the top of the tiles. I think maybe we weren’t supposed to really let it dry all the way, because now the tiles have a rock-solid gray film of grout all over them that does not come off. With scrubbing and scrubbing and scrubbing I can get a little of it off, but I am hoping there is any easier way. Is there some solution I could spread over it, or something? I hope we didn’t ruin our works of art!