The game Portal (released in 2007 by Valve) is known for it’s gameplay where portals could be used to teleport between different locations. Portals where rendered as virtual windows into the connected location with the well-known orange and blue rings around them. The game spawned a huge amount of memes, fan art and YouTube videos that used elements from the game.
Real life portals without trickery
The Kinect V2 is a sensor that can be used to record a 3D view of the world in real-time. It can also track users and can see what their body pose is. This can be used to perform head tracking and reconstruct a camera view into a 3D world as if looking through a virtual window. By using one Kinect for head tracking and another Kinect for reconstructing a 3D view, the virtual window effect of a portal can be created in reality. By using both Kinects for 3D world reconstruction and head tracking a two way portal effect can be achieved.
Hardware setup
In the setup used for recording the video two Kinects V2 were used. My laptop was connected to a projector that projected onto the wall. The PC displayed the other much smaller portal. Smaller portals hide more of the other side of the scene and allow for larger head movements. With a bigger portal you will run into the limitations in the field of view of the Kinect much earlier.
Instead of transferring the recorded 3D scene I swapped the Kinects and only transferred the recorded bodyframes through a network connection to mimimize latency. This limits the maximum range that the portals can be placed from each other. (about 7 meters when using USB3 extensions cables)
A portal opens as soon as a user is detected by the Kinect so proper head tracking can be done. For the video I used the right hand joint for controlling the camera so the viewer would experience what it would look like when head tracking is applied.
Holographic window
Holography is quickly becoming the buzzword of 2015. It’s getting harder to keep a clear understanding of what holographic actually means. (and yes I’ve abused the term too) I like the view that Oliver Kreylos has on the term holography. (See: What is holographic, and what isn’t?)
Since both worlds are already rendered in 3D it is a small step to add stereo rendering. For instance with a Microsoft HoloLens. This brings us closer to a holographic window.
Here’s the checklist that Oliver Kreylos uses in his article:
- Perspective foreshortening: farther away objects appear smaller
Check, due to perspective projection used in rendering - Occlusion: nearer objects hide farther objects
Check, due to 3D reconstructed world, but objects can occluded - Binocular parallax / stereopsis: left and right eyes see different views of the same objects
Check, when using a stereo display - Monocular (motion) parallax: objects shift depending on how far away they are when head is moved
Check, due to head tracking - Convergence: eyes cross when focusing on close objects
Check, when using a stereo display - Accommodation: eyes’ lenses change focus depending on objects’ distances
No
Natural user interface
Looking through a window is a familiar experience for almost everyone. Imagine being in a Skype conversation and having the ability to move your head to see who your caller is looking at or talking to (when it’s not you). A holographic window has the power to give people the feeling of being in the same space and allows for interesting new interactions. Anyone care for a game of portal tennis?
Leave a Reply
You must be logged in to post a comment.