Since the original Tron movie came out in 1982, hardcore gamers have been dreaming of stepping, body and soul, into their video games. Oliver Kreylos has been one of virtual reality's die-hards since 1985. Now, equipped with a trio of Kinect sensors and an Oculus Rift, he's actually accomplished what Flynn did in the movie — or pretty close, at least.
A combined live stream from three Kinect sensors, arranged in an equilateral triangle around Kreylos, is capable of generating an (admittedly glitchy) 3D image of him in real time. Armed with this video and an Oculus Rift, he then generated a first-person image of his own skin in a virtual environment. Adding computer-generated furniture, Kreylos was capable of creating a virtual reality that his brain immediately thought of as solid and real. While he was fully capable of walking through the virtual furniture, his instinct was to avoid it with his VR limbs. By stripping down the Kinect's video to a raw feed, Kreylos has even eliminated the latency that the Kinect usually doles out when running its skeletal reconstruction software.
Even though practically every video game you can buy today has graphics that put his live Kinect feed to shame, Kreylos believes that the true-to-life movements of his own arms, in his own clothes, far outweigh the better looks and imperfect movements of a computer-generated avatar where immersion is concerned. The whole system was set up on a single computer running Linux and boasting a fairly run-of-the-mill eight gigs of RAM, a Geforce GTX 770 graphics card, and an Intel Core i7 processor. If he continues to refine his 3D imaging system, Kreylos could very well someday give the rest of us a means to join him in a world of our own making.