They're called "Shaders"

Everyone is swooning over the darling of virtual environment engines, Crytek. Used for such things as Farcry, Farcry 2, and the infamous Crysis. The Crytek engine (now in Crytek 2 and recently announced Crytek 3) sports very impressive graphics for a game engine, and in many cases reaches into the realm of near photorealistic.





While the Crytek engine series is very impressive, there is a very simple explaination for why this engine is so graphically amazing. The secret is Shaders. HDR Shader, Bloom Shader, Depth of Field Shader, Motion Blur Shader, Atmospheric Scattering Shader and others all working in concert.

Add to that some lighting shaders and soft shadow shaders for Shader Model 3, and you begin to understand just what the Crytek engine series is all about. This isn't to say that the Crytek series is not impressive, because it really is. But running that many shaders simultaneously is obviously going to require serious hardware (as if you didn't know this already).

If ever you doubt this, simply turn on the shaders for SecondLife and you will see that it does have a negative impact for less than stellar hardware.

Breaking down the Crytek engine series a little further, let's look at the ocean system. Ocean surf, 3D water, refraction and reflection... Ocean Surf is a Shader. 3D water is easily taken care of by Projected Grid techniques, refraction and reflection are also shaders applied to the surface and under surface.

Add to that 11x Antialiasing on the hardware, and the environment looks fantastic, although you must admit that a majority of the greatness for Crytek happens to lay in shaders. Tesselation routines can take care of the terrain, and as for the trees and vegetation, start with high end tree models (we suggest from XFrog.com), and add PhysX or Havok Soft Body Physics to them. Now you suddenly have gorgeous looking trees with destructibility and reactive behavior to the wind.

Throw in Parallax Extrusion for textures, and now you have simple and more realistic 3D surfaces without building geometry to handle it. This would be the solution for things like the sand on the beaches or brick walkways.

Combine soft shadows on the GPU with Per Pixel Lighting, as well as a handful of other shaders (like Volumetric lighting) and suddenly everything looks photorealistic. But the real question is - if all of these "innovations" are just shaders, is access to the Crytek series of engines really worth $750,000 and higher? How about the ridiculous requirements just to qualify for using these engines?

Crytek has a commodity and has made a name for themselves in the industry as the company which brings you amazing landscapes and photorealistic environments. By doing this, there is enough hype around them to warrant the lunacy of being the "it" engine to use for a gaming or virtual world project.

Problem is, a majority of Crytek is simply using Shaders to achieve the photo realism. Shaders which just about any 3D Engine on Earth can easily use right now.

So the reality of Crytek is that it's an impressive engine, yet it's not special in any way other than the fact that most game development houses are currently too lazy to develop engines themselves or work on the shaders to use for their current engines. Crytek happens to be the lazy man's way out when developing, and is the quick and dirty way to jack up the hype around their project (by being specially selected to use the Crytek technology) as well as look visually stunning with little to no hard work.

Any hobbiest programmer can match the Crytek engine using something like OGRE3D. Just include all of the required shaders, define high definition content, and use high definition textures. The only real difference is that Crytek was the first group to go ahead and put it all together professionally.

In the end, $750,000 + just to license an engine which is essentially a bunch of shaders seems a bit overboard. But weirder things have happened in the game development world.

From The Front Lines

Some news to report from the front lines:

Adaptive Mesh Prototype

Essentially it's a tessellation routine for our planet system which reduces the mesh complexity based on distance. When the viewer gets within a certain distance of the planet, the system begins to resolve the landscape itself including mountains and terrain.

This is good for such things as a seamless transition from planet surface to space and back, aka: Google Earth style. Except that we plan on having multiple planets in the virtual solar system, as well as eventually many solar systems in a galaxy (continue scaling up to get the big picture).

Adding to this is the combination of Mesh Reduction LOD in order to handle the objects on these planets as you get within view to begin loading them. As a result, the mesh reduction LOD system should allow for a visibility range to the horizon without adverse effects on the rendering engine.

If you are an Active Worlds user, then this means the visibility menu is obsolete.

The next step is to figure out a dynamic tessellation routine.

Some Musings

After looking over the tessellation routine in action with a virtual planet the size of Earth, something has dawned on us that we didn't previously realize. An entire planet in A3D may be overkill for the standard needs of the average client/user.

If the largest "world" that ActiveWorlds has in their "universe" is the size of California, and they have yet to completely fill it ten years later, then the main planet of Andromeda3D may never be filled at all.

Which begs the question... is A3D simply too powerful? At the moment we're not going to question the power of the system, but we may have to offer planets that are orders of magnitude smaller than Earth to our customers.

A Recipe For Awesome...

One of the recent endeavors that we have been on here at Andromeda3D is to figure out a solution for one of the oldest problems in virtual worlds. That is, how to easily allow the user to make their avatar look like themselves should they choose to. One popular approach is to simply take two photos of the subject, one from the front and another from the side, and wrap a face map to a generic model of a head. While this is fairly quick, the results aren't so good in the end as the face map tends to warp and stretch to the incorrect mesh.

The obvious solution would be to have an algorithm analyze the photo and then mold the facial features of the mesh to fit the photo. As you can imagine this is no simple task, but with a bit of research it was discovered that there are definitely ways to accomplish this.

So we find a photo of an individual with which we would like to import into the system as a 3D representation of the face on an avatar, and we allow the algorithm to analyze the photo while actively morphing the model to match the features.

In our test, we chose a familiar face to work with -

If we take into account that this was the photograph imported into the system, obviously not an ideal front shot for 3D modeling, we can see that the result of the algorithm and mesh matching are indeed impressive, as shown below:

But this is not the end of the brainstorming and technology think tank session, by no means. The facial generation system also rigs the animation points on the model so that phonemes are also incorporated. The first thing this brought to mind was allowing the mouth to move realistically when speaking, but not on a post process sort of manner.

What if we marry this technology together with the VoIP system, then marry those two together with a Real Time LipSync system so that not only would the user have the ability to look like themselves in the virtual world by importing a single photo, but also have their virtual lips sync to their VoIP conversation as well in real time?

Now we're getting someplace. After a bit more research into this idea, we came across an SDK which allows just that - real time syncing of streaming audio to mouth animations based on phonemes. Not only can the avatar look photo realistic, but it can actually move its mouth properly when you speak. The downside so far is that the delay in audio processing is about 10ms, which in the grand scheme of things isn't actually that bad.

As for the face generation algorithm, on a decent computer with a good graphics card the process should take roughly 2 - 5 minutes to create the custom face and head. On slower computers (or computers with bad OpenGL Drivers) this process can take an extended amount of time into the 10 - 20 minute range for completion.

While the generation process may be slower on outdated computers, it still works the same way with the same results. So it's a matter of whether the user wishes to wait for the customization process to finalize or not. There is a definite up side to all of this in that once the face is customized to the photo, the file generated to recall the data is around 40 - 50kb in size and the calculations for mesh morphing are not required again (unless the user imports a new photo).

Then we add to this chain, yet another feature we've been working on. The Shader Skin process by which a procedurally generated skin texture is applied to the mesh as a first layer, then the face map is applied as the top layer, adjusting the opacity and color to come to a close approximation and create a seamless transition.

As for hair, again we're looking at Shaders for high end computers or premodeled hair geometry for slower systems. nVidia has been doing some interesting research concerning realistic and physX enabled hair in real time, so we'll be definitely keeping an eye on that.

Let's take a moment to recount our recipe for awesome -

1. Import photo which then automatically is analyzed to create a matching 3D Mesh for avatars
2. Apply real time LipSync technology to a VoIP stream
3. The facial animations of the model are already rigged and ready for use in the LipSync scenario
4. Real Time Shader Hair with PhysX
5. Procedural Skin generation to blend seamless between face and rest of body

Since the Digital DNA profiles of people are very small files, comparing a generated file to one on file already (such as Robert Downey Jr.) would be fairly easy. This would enable us to make sure people who are importing a photo aren't importing a known celebrity in the database who has requested IP Protection from Andromeda (Part of the VIP Program).

For the time being, this is definitely an avenue of research which we wish to continue as we put this virtual universe together.

- The Management