Unreality

#AugmentedReality Step into a virtual environment where virtual reality and reality merge. |

 

One of the many things that cross my mind when thinking about the future of virtual environments is whether or not any virtual environment will reign supreme in the future. I hear a lot about how virtual environments will redefine the future of interaction, or about the latest advancements, however lesser heard is how the true future of virtual environments will evolve.

 

While things like Second Life have a lot of merit and potential in the grand scheme of things, I truly believe the real focus for the future is in Augmented Reality. not the augmented reality we see today, though, because it is currently too limited to make a dent. I mean, we see augmented reality today and it is tied to this notion of markers for the camera to recognize and translate into digital data, whether it be some QR Code or a predefined picture to recognize.

 

Aside from that, it’s very limiting because we have this stationary camera with which to see the world through the eyes of the computer, maybe a web camera or similar. So in that respect we’re at a point where Augmented reality is really limited to stationary scenes or instances, maybe holding up QR codes to the camera and viewing the results through the computer screen.

 

But what if this Augmented Reality was unshackled from these limitations?

 

 

 

Imagine this, in real time, from a first person perspective.

 

 

 

I suppose we would have to first take a look at the current limitations and address them properly, or at the very least make some speculations as to how we are to overcome them.

 

Let’s first address the issue of marker based augmented reality. To me, it’s a good first step in the right direction but is far too limiting. In order to make this ubiquitous, we’d need to devise a method of markerless augmented reality and work from there. I’ve seen some really good steps in this direction from such things as HandyAR, and in recent weeks announcements from Sony have really peaked my interest in the arena of augmented reality. So let’s say that markerless augmented reality is a solid approach.

 

The question now stands as to how we unchain the view from a stationary aspect and make it truly mobile in nature.

 

I would imagine that such a system would be a lightweight visor, using things like OLED and maybe even a depth system like we see in the Nintendo 3DS to capture a reasonable depth video of the surroundings in a mobile form. But of course, this isn’t enough. The camera on the visor must also convey the depth of the surroundings, so we turn to 3D cameras like the one found in a Kinect to manage that, but probably with higher fidelity and resolution (which is a matter of technology progression).

 

So far we’re on a good track with this.

 

 

 

We’re beginning to see the possibilities.

 

 

 

Now onto the actual environment and markerless tracking. If we don’t have an actual marker any longer, how do we handle the positioning and environment?

 

Well, an idea has been in my head for a number of years (despite knowing how feasible it is) that could offer some insight. What if the unit had GPS? Well, we have an idea of how accurate the tracking resolution of a standard GPS is these days, and thus could create a virtual grid overlay using that data. I suspect the longitude and latitude plus minutes and seconds (resolution allowing) would be enough to define that virtual grid overlay. In the event it is not accurate enough, we can always do client-side approximations and tracking to compensate.

 

So we have a basic grid overlay, using GPS, and a visor to convey this Augmented virtual world.

 

Of course, this is still not quite enough, but it becomes a hell of a core framework to build on. Going forward, such a system would rely heavily on Wi-Fi for data, so it probably wouldn’t work too well outside of that sort of range, unless 4G data plans on cell phones could offer a solution (though to be honest, cell phone companies have an archaic business model and stranglehold on that system – charging for overages and per minute usage).

 

I think we’re getting somewhere though…

 

Let’s say, then, that the initial limitation would be within a Wi-Fi range (with advances coming later to alleviate this). So we run the bulk of the software and computation on a cloud system, and stream results to the visor for output. In this manner, we take the grid overlay information, the user orientation and position, plus the location of 3D real world objects (as recorded by the Kinect type 3D Camera on the visor) and we have a virtual landscape to start with.

 

Just as the visor is broadcasting your own position to the cloud server, the cloud server is keeping track of others with a visor system in proximity to you, updating your visor accordingly.

 

Alright, this is looking good so far.

 

Now, let’s say we want to create an item in the real world grid from our virtual options. I suppose having an inventory at our disposal would come in handy. Data objects are phantom for the most part but would respond the the user through physics or virtual touch (like buttons and such), and that would be programming/scripting to accomplish. However it wouldn’t be like pulling out a virtual chair and dropping it on the real world grid to sit on it. It’s not really solid, so we accept that these digital items are mostly there for informational or augmentation of environment purposes.

 

You may be noticing that my interpretation of the Augmented Grid isn’t a passive experience, like we normally see and expect from current AR. Sure we’ll have widgets like Weather apps and information displays, but those are just feeding you information on a one way street. I’m interested in a dynamic augmented grid where we participate and create openly.

 

Keeping track of spatial alignment and position for virtual objects in this augmented reality would fall under the combination of GPS, Kinect, and Cloud Software to calculate, so when we attach… say a virtual digital information sign to a building (a virtual notecard so to speak), the Kinect camera on the visor knows we are facing a real world object that is solid, the gyroscope and internal compass knows the direction, and the GPS knows the rough grid location where that data gets stored. Toss in an altimeter while we’re at it in order to give access to a Z location and not just X and Y.

 

Since this system is running mostly on a cloud architecture, we could include any number of virtual systems and widgets into our Spatial View. Maybe Skype calls with video and audio? hand gestures at this point become trivial for controlling the system (and subsequent programs) loaded into our visor via the Kinect camera and APIs for motion tracking. The same aspects wold also be used for manipulating the virtual overlay environment, grabbing and positioning virtual items, displaying an augmented keyboard overlay on a surface to type text onto a virtual notecard in 3D space, the list goes on.

 

In reality, all we see is a person wearing a visor headset.

 

There would be an overflow of data in the 3D world around us, and so filters would be employed to keep the stream relevant to our interests. Possibly hand gestures while facing locations (like restaurants, venues, or even other people) would allow us to access further information on demand.

 

Then there is the augmented self.

 

It may seem trivial, but we do this already with virtual environments in that we customize our virtual avatars in the manner with which we wish to be perceived. It would be no different in the Augmented Reality space, in that since we are broadcasting our virtual positions to the cloud architecture, the Kinect camera can overlay the virtual skeleton and track points, and then overlay a virtual avatar on the person’s real life body, controlled through real life movements in real time.

 

I believe at this point we’re getting dangerously close to the realm of being masters of our reality.

 

We could be walking down the street, in real life, wearing a visor and the view of the world around us would be vastly different. Other people would look like virtual representations, maybe avatars of all types. They could also have scripted items on them like we do in Second Life, and as such are represented in the Augmented Grid accordingly.

 

Imagine merging Second Life with Real Life, and then taking that a hundred-fold further with the amount of information and personal interaction access we’d have on-demand in spatial reality.

 

We would create games that ran in real life, through augmented grid space. Zombie invasions in our back-yard, Tower defense on our front lawns. Augmented Reality HALO games in the neighborhood park.

 

Augmented distance games against people around the world, on-demand.

 

Then there is a viable business use as well. Imagine being able to redecorate an entire house in real life with virtual furniture, painting all of your walls different colors in real time without touching a drop of paint. Want to see what you’d look like in a new pair of clothes? Try it on in Augmented Space. Just as such a system can project an avatar overlay, it could just as well be used to overlay a dress or shirt onto your augmented wireframe. Cloth physics of course would be applicable for that.

 

Maybe your avatar is also a hybrid? A real life self wearing augmented items?

 

Tour guides in art museums would be great as well, with audio and visual. Augmented art galleries, where you are in the environment itself or even in the paintings… who knows?

 

City walks would be phenomenal using such a visor system.

 

Want to see what a building will look like before it is built? Overlay the HD model on the AR Grid at the location where it is supposed to be, and get a feel for the presence and looks of the design in 1:1 scale.

 

This is the world I wait to live in. It is a world inspired by a dream I had years ago, where all of these things were not only possible, but commonplace. Experienced from a first person perspective it is absolutely wonderful, if not a bit confusing at first.

 

I’m sure this is long standing dream of many futurists… but that doesn’t stop me from dreaming :)