How can virtual 3D be used?

I read that there is a Sony game that features a head mounted display with a position sensor on the helmet that shifts the scene as the head is turned. I think the idea behind this is awesome.

Suppose this idea is extended so that not only will the scene change as the head is rotated, but also changes as you move. You could have an empty room filled with virtual objects. Now add sensors on the hand that provide a tactile sensation as virtual objects are touched. Provide the ability to lift and throw virtual objects. Finally, add a virtual jet pack that allows the person to move in 3 dimensions in the virtual world.

Is this doable? Might there be practical uses, like training surgeons on virtual patients? There are unlimited entertainment possibilities. Imagine a virtual jungle where what you see is determined by the path you take. We can take this a step further by allowing for group experiences by providing each person with an image of the others in the virtual worlds.