Requiem For Blue Mars

Written by: Will Burns [Aeonix Aeon]

Twitter: @DarianKnight

::: TLDR Warning :::

 

 

Ridiculously Short Summary:

 

Blue Mars is Broke. The CEO is gone and most of the team. They’re using the money they have left and the board of directors is betting the remainder of the farm on mobile/web technology with OTOY, because they can’t afford to develop a desktop native client any longer. Chances are the cloud rendering option will eventually bleed them dry and what is left of Avatar Reality will no longer be a reality. Read on for full details and synopsis.

 

Full Article:

 

In what can be seen as an unexpected move on Friday, Avatar Reality (makers of Blue Mars) announced their intentions to restructure the company and focus mainly on mobile platforms. This will, inevitably, come at the expense of the PC client many currently use, and over time the PC client will be phased out entirely in favor of the iOS (Apple) mobile client application and possible web client.

 

Of course, this would have come as a surprise had it not been for the simple fact that this was not unexpected.

 

Rather than speculating on the news of Blue Mars, I did what anyone in my position would do; I simply had a chat with somebody who actually works (worked) at Avatar Reality and asked what happened. Reasonably speaking, I had asked not because I didn’t actually know what had happened, but I wanted to hear it from him directly.

 

Here is what actually happened:

 

Avatar Reality has run out of money.

 

The investment capital has more or less run dry and without further capital, the powers that be (the board of directors) have decided to put what capital is left in the coffers toward Mobile and social media integration development, betting what’s left of the farm on OTOY cloud rendering. In August 2010, their plans for utilizing OTOY cloud rendering for Blue Mars did not come to fruition, and as a result the delay in implementing that solution caused the company to more or less fold. In honesty, when the CEO of a company resigns and takes a large portion of the team in a “restructuring” effort with them while the remaining team is focused on social media and web applications it’s a clear sign that… something… is…

 

Why does this sound familiar? Oh.. right… Second Life. *sighs*

 

That’s not to say that OTOY is at fault for this, because if the success of your company relies completely on another company to deliver a solution that your own company cannot provide (when clearly it is your company’s job to provide that solution) then the fault lies with your company for having a broken system to begin with.

 

While Avatar Reality still exists as a shadow of its former self, at this juncture it is now a matter of when and not if the company will completely deplete their remaining investment capital and run ashore with their ship. However, maybe the plan here is roughly the same as with Second Life?

 

You know… Facebook and twitter integration, a cloud rendered streaming web version, all in the hopes that some kind corporate giant will buy the company and reward the board of directors and team leaders for making bad decisions and blowing millions of dollars in investment capital on poorly thought out virtual environment architectures that could have been avoided by simply doing basic research. As an aside:

 

*Waves to Mr. Humble, the new CEO of Linden Lab*

 

Part of a prior conversation with Avatar Reality and myself roughly six months ago outlined this issue directly and stated that the servers alone will cost more money than can effectively scale at cost, citing bandwidth requirements alone in serving premade worlds in bulk download plus rendering and streaming on adequate PCs capable of at least offloading some of that rendering cost. While it is true that OTOY can effectively render and stream complex scenes over a broadband connection, the factor which was ignored was at what cost for that computation and bandwidth.

 

As it turns out, and rightly so, with or without OTOY in use, the cost of rendering and streaming Crytek level graphics costs far more than the company could sustain, and by all means more than any content creator was willing to pay to have hosted.

 

Which brings us back to the Mobile/Web decision.

 

With a PC client, there was at least the vague option that the client was running on a computer remotely capable of offsetting the rendering costs locally. After all, the Crytek Engine is nothing short of hardware intensive (to say the least). However, choices were made internally to stream the rendering server side at what can be quickly assumed to be more money than they were capable of making to offset the bills.

 

With a mobile platform in the works, that cost reduction no longer exists. Avatar Reality, in some twisted logic has decided not to choose a solution which would allow them to offset the costs but instead would inherently dictate that the full brunt of the costs be on their own heads. Can we say that the iPad, iPhone and related mobile smartphones are actually capable of rendering Crytek level graphics in real time?

 

Of course not.

 

Most PCs short of monster gaming rigs have a hard time using the Crytek Engine, and to assume that a mobile platform would be more capable than a high powered personal computer (whether an Apple or PC) is simply ridiculous.

 

So the mobile option now involves not less centralized computing to lower costs but more centralized computing, bandwidth and rendering to vastly underpowered devices. It seems plausible that having one foot in the grave at Avatar Reality isn’t enough to invoke common sense within their board of directors.

 

As noted with SecondLife Skylight, cloud rendering and streaming does in fact work, however the bandwidth used to stream that information (not to mention the costs involved with cloud rendering) are something to be alarmed about. In tests done by various users of Skylight, the standard bandwidth used over the course of a 60 minute session was 1 GB or more.

 

I’m not aware of a mobile provider that can or will support 1 GB of data transfer per hour on a data plan, and of those who may allow such a thing, the “unlimited” plan would surely be brought to its knees fairly quickly, causing massive overage charges to the user in question. Then there is the question of who exactly is footing the bill for the cloud rendering and streaming of that data to begin with (aside from the massive data charges for the mobile user).

 

So let us assume that a web based system (or mobile platform) based on cloud architecture is really aimed at WIFI tethered Mobile devices or websites where the average user may as well be using a personal computer (again, Apple or PC). But in this case, the question then becomes not if this is possible, but rather why anyone would not simply want to use a native desktop application for this purpose and gain more benefits from doing so.

 

In the case of Skylight, I can understand the plan to somehow create the ability to make Second Life into an easy to use application for Facebook, of course stripped down considerably in ability. Regardless, we’re still looking at much more bandwidth in use for a fraction of the functionality and possibly additional costs server side for rendering and streaming that information.

 

So who will foot this bill?

 

Surely it won’t be the content creators of Avatar Reality. The cost is simply too high and the returns too small to consider it an adequate ROI.

 

Maybe Avatar Reality will translate those costs to subscriptions for the end-user? Again, the cost is too high to be worth it to the end-user and the returns are still too low to justify it.

 

In the end, Avatar Reality will be footing the cost of this platform, with marginal (at best) returns from content developers and users. In no way will those returns remotely close the gap for the costs.

 

Looking Back at the Future

The really interesting part about all of this is that in order to see the future of Avatar Reality, and subsequently Blue Mars (or any virtual environment today), we need not look into the future but instead look to the past.

 

When I originally used Blue Mars in the early beta, I had this nagging feeling that it seemed familiar to me… like I had seen it before. After a few days of thinking about it, I finally realized why Blue Mars seemed familiar, and I was correct in thinking I had seen something like it before; as a matter of fact I had seen something exactly like it before.

 

Worlds Inc + Crytek = Blue Mars

For those of you in the industry seasoned enough to remember, Worlds Inc was one of the grandaddys of Virtual Worlds platforms in the early 1990s. The WorldsPlayer client was downloaded and each individual World had to be downloaded separately and in advance prior to having access to it. Those worlds were also pre-made and “published” as scenes. The client came with a starter world (welcome area) and had a “map” of other worlds available to visit (after varying download sizes and installations).

 

At this point, there really is no differentiation between whether I’m describing Worlds Inc or Blue Mars anymore. The difference, though, happens to be in whether or not Blue Mars managed to learn absolutely anything from history.

 

In this case, the answer is no.

 

Worlds Inc was in a position where they had to think creatively in order to minimize bandwidth because the average user was lucky to have a 33.6 Modem let alone a 56k Modem. Blue Mars, deciding that they wanted to look good, decided to license the Crytek Engine and then have designers create 50 to 500MB scenes (not to mention a 300 + MB client download) that needed to be preloaded in full and installed prior to entry. Instead of trying to find ways to minimize bandwidth usage and graphics rendering requirements to something a “normal” computer could actually handle, Avatar Reality saw fit to create a system in which the only possible way to utilize it is to have a veritable supercomputer cluster pre-render the entire program for each user and stream the results over broadband.

 

Is it my imagination or has the virtual environment industry become mentally handicapped?

 

Instead of learning from past mistakes, and doing everything in their power to avoid them, the entire virtual environment industry is stuck in some weird stupidity loop, literally doing the exact opposite of what they should be doing as demonstrated by every possible prior attempt in the industry. Even if we look at web clients like web.alive/Avaya, I can point to it being a rehash of VRML and X3D in the 1990s, and we already know what happened with that.

 

None of this is new or innovative.

 

In 1990, which is now 21 years ago, the solution to this madness was given. Every year after that, a new virtual environment springs up, doing the exact opposite of what was said, and somehow is left in surprise when millions of dollars in funding dries up and they are left with a company (and virtual environment) that has failed.

 

Remember the early 1990s? VRML was all the rage, and there were companies springing up all over to offer some sort of Virtual Reality software or service. Blaxxun Interactive, ActiveWorlds, Worlds Inc, you name it… Virtual Reality was new and exciting! It was the future of communication!

 

And then VR died.

 

Of that wave in the 1990s, I’d say that ActiveWorlds is still around but barely alive. Worlds Inc is apparently responsible for Utherverse and Red Light Center which is essentially a XXX pay per play virtual environment (and still terrible quality by today’s standards). I haven’t the slightest clue what became of Blaxxun Interactive, but truth be told, we don’t hear much about that company any more (assuming they even exist).

 

What did we learn from this wave? Apparently nothing.

 

In the early 2000s there was yet another wave of virtual environments and services, There.com, SecondLife, Blue Mars, Kaneva, web.alive/Avaya, and at least half a dozen more (Open Croquet, Cobalt, Wonderland, Project Darkstar, Proterra, Proton Media…) and the weird thing about this is: The people using, and those responsible for, these recent systems have amnesia.

 

I remember reading a statement by Rivers Run Red claiming they were the first business in the Metaverse, and obviously I laughed, but then I got very worried. A company like RRR who is supposedly representing corporate interests in the virtual world space, should be smart enough to never have made that claim, and yet they did. Much like listening to Philip Rosedale claim that Second Life is the Metaverse and that Metaverse really means “Metaphysical Universe”.

 

In 1990, the solution was given by two people to all of this madness. Chip Morningstar and F. Randall Farmer, authors of Lessons Learned From Lucasfilm’s Habitat. Strangely enough I had asked Mr Farmer about Linden Lab and he informed me that he was actually called in as a consultant in the early days, and not surprisingly, ignored.

 

The answer they stated: Centralization is the bottleneck and does not scale cost effectively. A decentralized approach is needed in order to effectively build a system capable of scaling to millions or hundreds of millions of simultaneous users.

 

So what has every system since 1990 done?

 

They’ve all become increasingly more centralized, and each company has eventually met their demise as a result, or have become footnotes in the history books, forever ignored and forgotten by the next generation who apparently enjoy making the exact same mistakes.

 

Avatar Reality had ample warning, however, that this was going to happen. I know this because I personally warned them. Just like I warned that Second Life would have similar issues. And, strangely enough, just as I warned Activeworlds that their system would need a complete overhaul to remain both competitive and relevant.

 

In each case I was told I was wrong, and while I watch ActiveWorlds implement my proposal list four years later, Avatar Reality hits a brick wall and runs out of money, There.com ceases to exist, and Second Life “restructures” and hemorrhages CEOs… well…

 

They say that hindsight is 20/20, but you’d think that there have been enough mistakes in the past to convince people today not to repeat them.

 

For somebody that is constantly told he’s wrong, I seem to be getting an awful lot right. The only difference is, it’s not costing me tens or hundreds of millions of dollars in the process.

The Sounds of Silence

If you think your Dolby SRS sounds amazing, you’re sadly mistaken.

 

 

When faced with the task of creating a fully immersive virtual environment, quite often I have found the results to be lackluster at best. In terms of SecondLife, it is not the textures or details for models (prim or mesh) that concern me, but something more important.

 

Let’s Set The Stage

 

While we can look at textures and always say that higher resolutions are better representative of the image involved, there exists a static approach to this process which will forever limit our ability to truly see an exponential increase in the fidelity over time. 512x512 resolution or 1024x1024 resolution are the sorts of limitations that are inherent with graphic cards cache and so we are forever limited until those numbers increase.

 

The same can be said about the detail levels of mesh and prim creations, in that the polygons there are (vertices, etc) the more calculation it takes to render that item in three dimensions, and thus in an environment such as SecondLife (I use SL as an example quite often), we’re stuck with lower end mesh and prims with orders of magnitude lower fidelity.

 

However, this is not entirely true, in that we aren’t completely limited by the hardware in either case. Instead we are limited by the implementation of how those hardware components are utilized. In the case of textures, it is not the limitation of 1024x1024 resolution which exists, but instead the limitation that the GPU can only comfortably (in most cases) handle that resolution in cache at any given moment. We would assume then that the two statements are the same, but there is a distinct difference in subtlety.

 

In the area of procedural textures, a much higher fidelity of texture can be synthesized with algorithmic methods, creating the ability to scale upward dynamically in resolution while not losing fidelity. While the texture itself may be (in theory) 32,000x32,000 resolution (in 2kb), the idea here is that we don’t actually need to see the entire resolution at the moment, and therefore should stream only what we need through the GPU. Procedural methods for textures are much better for handling this concept of perceived hyper-texturing than any static texture could ever hope to achieve.

 

 

Similar developments occur when we scrutinize mesh and prim based three dimensional constructs (3D Models), in that the detail level of the models imported into a scene are not necessarily representative of the capacity for fidelity. In our case, a 3D Mesh is merely a base fidelity which can be highly augmented for further clarity and detail through dynamic and algorithmic means such as Tessellation, Parallax Extrusion, Bump Mapping, Screen Space Ambient Occlusion, and more.

 

In terms of mesh detail, Tessellation comes to mind as a manner by which to give much more detail to models (and textures) through GPU algorithms. Take for instance the tessellation routines built into DirectX 11. Essentially what this sort of algorithm does is intelligently subdivide quads for further detail based on camera proximity, and often times the results can be quite amazing.

 

For those of you who were just derailed from that bit of geek speak, simply put: The closer you are to something, the more detail the GPU gives the models. In combination with Parallax Extrusion algorithms, we can take flat textures and make them 3D while adding further detail to models in virtual space where it did not originally have such a level of detail.

 

Again, we’re utilizing algorithmic methods (which are dynamic and not static) to augment and greatly enhance the detail of our virtual worlds.

 

 

Hardware Tessellation with DirectX 11 (UniEngine)

 

 

Of course, with Tessellation, we’re not going to suddenly find infinitely more detail where there was none. But at least it’s a beginning to what I believe will be a future trend for virtual worlds. Dynamic and procedural methods by which to automatically augment fidelity into the future. Let us not forget that these same sorts of algorithms can be deployed for things such as dynamic hair and clothing using a Physics Processing API. In SecondLife, the idea of amazing hair and clothing must surely appeal to a wider audience (mostly because I assume half of SL is in the fashion industry and the other half are partying in nightclubs).

 

 

Physically simulated clothing by CCP using NVIDIA APEX

 

 

Is It Real, Or Is It Memorex?

 

Now that I’ve outlined the two basic components of an environment, and how they can be made drastically better through different approaches to their use and creation, I’d like to focus now on the heart of this article and why you are reading.

 

It has been my contention that the reason most virtual environment spaces in SecondLife blast music at their locations is to avoid having to create a proper soundscape for their environment.

 

Audio is by far the most underutilized aspect of a proper environment, without it we simply move through spaces that are disturbingly quiet (like a silent movie) or are bombarded by loud music in order to cover the fact that in reality we (as designers) are piss poor at making a convincing environment which is capable of truly immersing our participants.

 

There is always exceptions to the rule with this, and there have been fantastic and breathless implementations of complete environmental immersion in SecondLife on occasion. However it still needs to be said that the nature of SecondLife will always limit the ability to actually make that environment convincing. Ten second audio in single channel with a low bitrate just isn’t going to cut it for creating our virtual realities in the future.

 

When taken into account, we would obviously think then that higher bitrate and stereo audio files would be the solution, and we would only be partially correct.

 

I truly believe that in order to create a completely immersive soundscape, we need to implement an audio solution that is better than stereo and high bitrates alone. Of course, then, we must be talking about Dolby Surround Sound right? Not exactly.

 

For virtual environments, I believe it is safe to say at this point that the use of headphones is commonplace. More often than not we are using our computers with a pair of headphones covering our ears, and most of the time that potential is horribly wasted on single channel audio effects or stereo music streams.

 

The human ear simply does not hear like that, and to make matters worse, the bitrate (quality) is absolutely crap to begin with for the source files. The ear is a wonderful thing, but hearing itself happens in our head. Your mind is possibly the most powerful audio processor ever known, thanks in large part to its ability to differentiate spatial awareness based on stereo input.

 

Cetera, which is what this amazing ability is called, is the mind’s ability to perceive the minute difference in arrival time between left and right ear for a sound and use that information to tell you where that sound is coming from in full spatial awareness. This is also something that can be implemented in standard stereo audio using a technique known as Binaural audio.

 

Depending on how the audio is to be created (or processed) you can simply record the source file in a manner which would create a binaural stereo recording, or you can incorporate the means to translate audio in a game environment on the fly using an API like Ghost which incorporates the Cetera Algorithm (among other techniques). Binaural alone isn’t the end of the line either when it comes to capturing spatial fidelity. If we take into account the proposed findings of Hugo Zuccarelli, then there may be an auditory interferometer which creates a sort of ambient reference sound for even higher clarity in interpretation in the mind.

 

In that case, we’re talking about Holophonic recording, which is a trademarked method of Binaural recording with additional methodologies to greatly enhance the output and spatial perfection. Of course, the claims of Holophonic audio are disputed by pretty much half the world while the other half swear by it. whether or not Hugo Zuccarelli really managed to create the equivalent to the audio hologram will more than likely remain in dispute for quite some time. What we can say for sure is that this is the same guy who also managed to create distortion free speakers and microphones that have zero feedback when held up to a speaker.

 

For now, we’ll simply focus on the undisputed claims of binaural audio.

 

Fiber Optic Head Trip

 

For this portion of the post, I’d like to direct your attention to the nearest pair of headphones. This is, after all, an interactive sort of blog and today we’re taking a field trip of the mind. In order for this to work, you need to be wearing a pair of headphones.

 

As we’ve discussed so far, Cetera is when your mind is taking into account the subtle differences between the arrival time of sound between your ears. This is actually quite specific and plays a massively important part in recreating spatial audio.

 

Got those headphones on yet?

 

Time for a Virtual Haircut

 

Alright, so this is a quick demonstration of binaural audio in action. For many, Luigi the Barber just made you cringe with a pair of clippers but also informed you how and why binaural audio works. As I said before, in order to make a more believable environment we need to explore methods by which we can enhance all aspects. At the moment, audio is the lowest hanging fruit to tackle, as it wouldn’t require novel techniques or incredibly expensive equipment to produce.

 

The problem is, binaural audio is recorded on a dummy head like the one below:

 

 

 

Well, not entirely a problem if you are recording audio intended to be heard from a single point of reference. But what happens when we are trying to recreate spatial audio in a dynamic environment? In SecondLife, there is “spatial audio” but I’m not convinced it is entirely useful. As far as I can tell, the spatial audio is essentially panning left and right channels (at best) with volume and doesn’t really convey spatial depth like binaural audio does.

 

So what we need is a method of dynamic Cetera calculation for three dimensional environments in order to allow the sounds to properly convey in a changing vantage point.

 

Luckily such algorithms exist, and are being refined for better accuracy in real-time. You can check out the recent launch of Ghost Dynamic Binaural Audio and get a better understanding of how this sort of technology is useful overall.

 

In the meantime, whether or not you have access to dual channel binaural microphones for recording, you can still keep in mind that creating a believable environment in the virtual world requires attention to detail with your audio. If the visitor cannot close their eyes and become immersed in your sim, then it’s time to go back to the drawing board and make things better.