Showing posts with label Content. Show all posts
Showing posts with label Content. Show all posts

Jan 19, 2012

Dreams in Digital

A discussion about the premise of Copyright, IP and Artificial Scarcity #SecondLife

 

The premise of Artificial Scarcity is, at the core, flawed at best and woefully inadequate at worst in the context of the digital age. It is no secret that copyright and intellectual property is gravely threatened by the digital culture we live in today, and increasingly so at an alarming rate of advancement. Such is the nature of the Internet to effectively circumvent censorship and treat all attempts to censor as damage and route around it.

 

 

android_female

 

 

After all, the Internet is a digital system that was at its very core designed to withstand a nuclear war and come out relatively unscathed.

 

What is most disconcerting is not that the Internet itself is nearly impervious to the worst attempts of censorship, but that the paradigms that fostered the creation of this system are no longer relevant to the situation at hand. Worse, still, is that this understanding of distribution of materials and information in the digital age is entirely counterproductive to the existing paradigms which create these media forms.

 

I can give a quick lesson on the past ten years or so of innovation in this digital culture, but I’m fairly certain most of us already know the drill. Peer2Peer came about in systems like Napster, and the copyright and IP industry lost their nut trying to kill it. All that managed to do was throw fuel on the fire, leading to more advanced networks of distribution and even after those legal battles ensued – first against the networks (which by law should have been protected under safe harbor) but later against individuals as young as twelve years old and as old as your dear grandmother in what amounted to a legal paperstorm.

 

Astronomical fines and penalties, legal cases that no individual could possibly afford to protect themselves against, and what turned out to be further fuel on the fire of piracy in the form of the invention of Bittorrent.

 

Clearly the attempted enforcement of artificial scarcity within the confines of a system that does not acknowledge scarcity at all is likely to lose, or make the problem worse.

 

There are very interesting insights from people like Jaron Lanier who believe that artificial scarcity is a necessity in order to give digital experience value and meaning again, but I will have to disagree with his assertion (at least in part). When it comes to Jaron, I rarely disagree with his vision – but this is one of those topics where I have to respectfully shake my head.

 

It’s called artificial scarcity for a reason, and that reason is because it is an artificial construct in a system specifically designed without scarcity in the form of digital media in all forms. When music was simply on CDs, there was natural scarcity in the availability of the content in physical form, but in the digital world it doesn’t cost anything to copy data. There is no physical cost in duplication, and the very least of the costs is relegated to the production of the original media.

 

But that’s where the costs really end. Unlike pre-digital days, the costs no longer carry over past the production cycle, and such things as marketing and distribution are freely available, if not exponentially so and for free. Even the production cycle has a diminishing cost, as we can see with DJs, Mashups and Youtube.

 

Nobody has to pay a distribution center to get airplay on the radio any longer, or produce physical compact discs for the record store shelves. Terrestrial radio airplay isn’t as important as it used to be in the face of countless internet radio stations, satellite radio subscriptions and more.

 

This is the dilemma.

 

It’s a matter of old school thinking versus new age. In a world that became accustomed to the natural scarcity of products, and literally based their entire business model on that natural scarcity and the control of the distribution channels from start to finish, it must be terrifying to suddenly lose that control – and so we see a constant battle between copyright and IP holders (RIAA & MPAA) and the “Pirates”, culminating in toxic legislation such as SOPA and PIPA, as well as the attempt of leveraging $75 Trillion dollar fines (RIAA vs Limewire).

 

This isn’t a new problem, however. This goes back to the printing press itself, wherein the same arguments were leveraged and fought tooth and nail. It carried on further with the invention of BetaMax and VHS, the Cassette Tape, Blank CD Media, and the MP3 Player itself (Roxio), and even into the right to use the MP3 Format altogether (Thanks, Fraunhofer, for the legal battles).

 

The thing about this history is – no matter how much we cry and complain about the damages that a new technology will bring to an industry in the form of piracy, those fears never seem to materialize. Even more interesting is the seemingly opposite effect that piracy has had on the music and film industry, with the record profits of those industries plain as day despite their constant arguments that piracy is hurting the industry and costing them hundreds of billions of dollars a year. Uproxx provides an excellent insight to this by giving an overview to the situation (further reading)

 

Pirates are your friends (Yarr!)

 

Yes, I just opened up a massive can of worms with this statement, but it had to be said. Given the industry of intellectual property and copyright, it can be shown time and again that the greatest harm to the copyright and IP industry are themselves.

 

Let’s break this down logically…

 

Modern piracy is simply the enactment of a highly efficient and targeted distribution system without any centralized control or overhead of cost. We can’t necessarily claim that piracy is stealing, because theft is the removal of the original instead of making a copy. Piracy is copying, plain and simple. The only difference today versus prior to the Internet is that there was a physical component which acted as the barrier to entry – Cassette tapes, blank CDs, VHS tapes, etc. and those media forms came with an additional blank media tax to compensate for perceived losses to the industry. The only reason that piracy is equated to theft is because of the false assertion that every file downloaded is automatically a lost sale.

 

If we were paying attention, we’ve seen this claim for as long as these industries existed – case in point:

 

 

piracy02 

 

 

Clearly this turned out not to be true, as the recording industry continued to make record profits and even today they continue to post profit gains and not losses. The only losses they are claiming are entirely imaginary losses based on fabrication.

 

We could make a convoluted claim that even the unauthorized copying is costing the owners of the copyright and IP the premise of future revenue which is entirely estimated, and based on fuck all, as seen with the RIAA statement that they believed Limewire owed in damages an amount that exceeded all the money in the world. The RIAA and MPAA can spew out all the make believe numbers they want, but it’s not based on anything substantial. It’s a lot like me saying that because people read this blog I estimate the damages from lost revenue to be 400 trillion dollars a year. I’m just pulling a number out of my ass, and totally out of context to the total situation at hand, much like the RIAA and MPAA does regularly.

 

Zen and the Art of Piracy

 

So how do we effectively stop piracy? I’m the first to proclaim that protection of intellectual property and copyright is something that needs to happen – despite my advocacy of piracy in the same breath. It’s really about looking at the bigger picture and actually taking the time to see the widespread implications past narrow interests of the old paradigm.

 

 

pirates_profit

 

For instance, think about all the money spent in the promotion and distribution of media by the RIAA and MPAA. More specifically, let’s look at the process for the RIAA wherein they spend an astronomical amount of money to A&R representatives in order to essentially buy a spot on the radio to play a new single for promoting an artist. This is a regular occurrence, and it baffles me.

 

When I was younger, my highschool actually had its own radio station and radio communications was one of the classes offered. Of course, I was enrolled in that class and managed straight A’s (45 days away from having my radio communications license to work as an on-air personality). Even then, the process baffled me due to the no solicitation rules. The station manager would have us regularly listening to new music coming in, and checking to see if it violated any of the rules, but that list of music always came from some A&R representing a label, whereas local bands trying to get heard would be routinely turned down. As DJs, our hands were pretty tied, as the station manager would be approving the playlist and helping to compile them.

 

Even if we were approving music on the list or disapproving it, most of the time that didn’t even matter. Those songs would be inexplicably stricken from the list when we approved them, or oddly added to the playlist when we as DJs decided the music was utter crap by consensus.

 

It more or less came down to who was willing to pay more money to get a band heard and played, and not necessarily based on merit. 

 

The apparent point of this process is to get an artist’s material heard by a large population of listeners, and they treat this process as a marketing expense, yet if we turn the tables and look at piracy – this is exactly the sort of distribution and outcome that Bittorrent has without the overhead. Interestingly enough, piracy is based entirely on merit and not who has the most money or connections for marketing and distribution – a stark contrast to how it used to work. Today, that local band (Garageland) wouldn’t have to be bending over backwards to get their music heard, and instead they’d simply make an official website for the band and distribute their own music via Bittorrent as a marketing loss. Those local bands would simply exist on their own merit and ability – which I always felt was how things were supposed to be.

 

So let’s call this what it is – it’s the inability of the industry to differentiate between personal piracy and commercial piracy. Commercial piracy is the problem and by all means should be fought or at least intelligently channeled, whereas Personal Piracy should actually be encouraged to no end.

 

The moment we stop trying to lump together personal piracy and commercial piracy, the faster we realize the benefits that personal piracy gives us. This idea that personal piracy costs the media industry hundreds of billions a year is absurd (as absurd as the assertion that it cost the RIAA $75 Trillion dollars in damages from Limewire) – because if anything it is actually increasing the revenues through widespread proliferation of new media at a cost that is exactly zero to the distribution channel (and subsequently to the media holders). What scares them the most is that media is now based on the idea of a meritocracy and not who has the most money or connections.

 

Personal piracy is good as long as we’re willing to call it what it is in proper context. The best course of action for the media industry going forward is to literally, and without hesitation, pirate their own media and make it available at a high quality to the masses.

 

 

stress-suit-man

 

 

Why on earth would I suggest this? I mean, I just offered the equivalent to the digital media apocalypse as the solution to the digital media apocalypse.

 

Natural Scarcity

 

This brings us back the idea of artificial scarcity versus natural scarcity. Natural scarcity was the model that the media industry was originally built on and it worked absolute wonders. It wasn’t until the introduction of mass reproduced media that they began to panic and invent methods of artificial scarcity to sustain a model that is based on natural scarcity. Natural scarcity was the sold out concert you couldn’t get tickets to, artificial scarcity is deliberately limiting the amount of CD’s pressed to inflate worth arbitrarily or (in the digital age) creating convoluted DRM schemes to restrict the usage rights of agnostic digital media.

 

The future of media is authentic experience. Natural scarcity is something you cannot reproduce in a digital form, and thus holds intrinsic value over time. You can film a concert, and maybe sell the t-shirts online, but you can’t pirate the actual unique experience of actually having been there. You can create DRM and other artificial scarcity methods to continue propping up your old business models, or attempt to force the old business methodologies on new media, but the sad truth is – artificial scarcity in an environment built without scarcity is a costly and failed endeavor.

 

In the movie industry, I’ll just cite all the DRM and Ultra-Violet Digital lockers having no effect on Bittorrent distribution except to make it more popular. However, I’d also state that all the unskippable garbage and advertisements on those DVDs are also a major factor, just like advertisements in between and during shows on television send people off to pirate the more authentic experience of the DVD version off of Bittorrent.

 

Physical goods and actual experiences that are unique are the new media paradigm to embrace, while the old media paradigm (currently being propped up desperately by artificial scarcity tactics) should be relegated to marketing channels in order to promote the natural scarcity of unique experiences and sales. Sure there are people out in the parking lots selling knock-off t-shirts for $5 out of their trunks, but the only reason they manage to make a killing is because you’re charging $10 for a cup of beer, $5 for a hotdog and $40 for a t-shirt inside. The problem isn’t the counterfeiters, it’s your inflated sense of worth and douchbaggery to the honest customers and fanbase.

 

Instead of writing piracy off as a revenue loss, we should be saying it is a marketing and proliferation surplus without additional cost. We’re actually saving money through personal piracy, if we’d actually figure out how to embrace it and wean ourselves off the artificial scarcity paradigm, we’d see that it’s actually helping far more than it hurts. As far as counterfeiters – actually curtailing them has less to do with trying to stop them and more to do with stopping yourself.

 

Telling a Story

 

Remember when VH1 had the Storytellers concerts? Only a certain amount of people could attend those intimate concerts, and there was far more natural scarcity for the experience than listening to it in MP3 or watching the recorded show on VH1. There’s a level of abstraction there, and anyone who watched those VH1 Storyteller episodes has at least once thought to themselves how amazing it must have been to actually have been there.

 

 

Authentic experience is the new reality of media | Greenday at Storytellers – Extraordinary Girl

 

 

Hopefully you’re starting to get the bigger picture now. VH1 Storytellers was (and is) a unique experience, an authentic experience, and not easily reproduced. It had intrinsic value – far more than the DVD version or the MP3 Soundtrack. In this manner, we foster the piracy of the VH1 Storytellers albums online, and even offer the videos online without restrictions (but maybe advertising is alright). The end result is you are utilizing the piracy aspect of a lesser experience in order to strengthen the value of the natural scarcity of a greater experience. You can always offer a pay what you want model as well, but you should never be under the delusion that it won’t be pirated for free after a short period of time.

 

If you want to offset that personal piracy as marketing loss, sit down and figure out how to charge for live streaming access to that experience, as well as the much more valuable experience of the tickets to attend that experience in person (since it’s a limited audience). These limited experiences should be more available as the revenue source, creating a value based on natural scarcity and access instead of artificial scarcity. The more authentic the experience, the more it is actually worth.

 

It’s always much better to attend a Broadway show than to watch it online, but if you were streaming it online for free, you’d drastically raise awareness (the point of marketing) and increase the demand for the personal experience of actually attending that play in person as an authentic experience.

 

The other benefit to pirating your own media is that you are immediately sabotaging the bigger threat of commercial pirates who are trying to sell the media illegally. Why would anyone buy what they can get for free? More importantly, what is left to pirate commercially is something that cannot be pirated at all – the authentic experience. Authentic experience creates loyal customers – which is another lesson for Social Media.

 

I actually applaud Deadmau5 in this context for the way one of his recent concerts was handled – it was streamed live on Youtube for the first hour (maybe half hour) as a preview and then turned into a pay-per-view sort of experience. However, this doesn’t entirely grasp the full potential of unique experience or utilize it to the fullest advantage. It’s a good start, but could definitely have been done better.

 

Sure, the ideal here for the limited access to a live streaming concert is still considered artificial scarcity, but it is based on natural scarcity at the root and not just arbitrarily applied as we normally see. For instance, in the context of the Deadmau5 concert, it would have been better to stream the entire concert for free in SD quality and make the HD stream a pay per view stream. The first half hour would have been an HD preview of the live concert, switching to the SD quality free stream afterward unless you had already paid for the stream access in advance.

 

Again, we’re going on the concept of pirating your own media for your own benefit, and writing it off as marketing expenses instead of lost revenue through piracy. It’s a balance between marketing and natural scarcity, creating additional value in a digital product or unique experience.

 

Who got this brilliantly right?

 

Pomplamoose

 

This is how you pirate your own music for marketing benefit. Take note, media industry.

 

 

Even better would have been if those concerts were streamed using not just high definition audio, but a Binaural audio stream for spatial audio fidelity. One step further would have been to offer that binaural stream in conjunction with 360 degree video so the viewers could pan around in real time with full spatial audio.

 

This is a massive step up from the lower definition of a simple SD stream for free, and something that constitutes a unique experience worth more money to the fans. The more authentic the experience, the more value it has in a culture where the lowest common denominator is mass distributed media.

 

And that’s the point, really.

 

It’s not piracy we should be afraid of, but the clear indication that non-authentic experience is nearly worthless in the digital age. It is this understanding that should be shaping our decisions going forward concerning digital media, copyright and intellectual property. Authentic experience has value, and that value can be used as marketing to sell more authentic experiences with increasing tangible value. The very fact that this point is being ignored on the whole is far more troubling than the specter of piracy.

 

Virtual Environments

 

Which brings us to the idea of virtual environments and the idea of prosumer culture. In a virtual environment where the users can create anything they want, it is no surprise that they more often choose to create what is familiar – as in brands that exist to them in real life. Companies have done an excellent job in marketing their brands into ubiquity and pop culture, but seem to be utterly confused when the ultimate result of that comes to pass.

 

Of course people want Coca-Cola in a virtual world and any number of other familiar brands, and they aren’t in a position to have to wait until such time as those brand owners decide to give it to them. This creates a conflict of interest, because we’re trying to apply the artificial scarcity rule to a virtual environment that (much like the Internet) has no such constraint of scarcity.

 

The current accepted practice is to file DMCA notices and have the offending content removed, which is a direct violation of the Streisand Effect, having the exact opposite of the intended outcome. Agnostically, we look at media overall – whether it is an asset or an MP3, and we should already know that it’s not in our best interest to call it piracy, but instead call it marketing.

 

In the end, what is more important to the end-user is authentic experience. As long as you control the authentic experiences, the non-authentic experiences of lesser digital value should serve as marketing in order to support the authentic experiences.

 

Case in point – let’s go back to Coca-Cola.

 

The proliferation of unlicensed digital representations of branded merchandise in a virtual environment like Second Life constitute the lower quality and less authentic experience, and as such act as the user generated marketing and advertising for the authentic experiences which they will gladly pay for in real life. Accordingly, the Coca-Cola merchandise in a virtual environment, generated by the users themselves is the best viral marketing the company could ever hope for, because it creates further brand loyalty (and is in fact a display of that loyalty) which in turn leads to the authentic experience of actually drinking Coca-Cola.

 

Even if those virtual representations are being bought and sold, it is in the best interest not to place that into the category of commercial piracy in that the users themselves are showing a willingness to pay to be advertised to on a wider basis than you could ever hope to achieve had you spent a few millions dollars on a 30 second Superbowl ad.

 

Does this necessarily mean that you should be relinquishing your right to the brand? By no means! It’s more about how you choose to handle that brand in context to the environment of experience. Much like Youtube offers a ContentID matching system for videos, virtual environments should offer something similar – wherein a ContentID match wouldn’t necessarily mean the content is removed, but instead simply labeled as being owned by a certain content or brand owner.

 

This follows the general rule of sanctioned personal piracy as marketing. If a content creator exceeds a certain amount of money in a virtual world from the sales of a branded item, then it should trigger an automatic percentage paid to the brand owner as compensation (a virtual item license fee).

 

In this manner, the brand owners actually make some money on what normally is seen as purely a piracy loss, instead treating it as pure marketing and licensing gain across the board. It also costs a lot less in legal fees, DMCA and litigation because you aren’t in a mindset to try and stop the actions but instead finding ways to foster it to the extreme for your own (and the prosumers) total benefit.

 

The end goal should always be promoting authentic experiences and fostering additional value in things which cannot be pirated, through encouraging personal piracy and brand loyalty to the extreme.

 

This doesn’t mean, however, that DMCA takedown requests aren’t needed. In context, it should be treated as the absolute last resort instead of the first. Explore all of our options first before we invoke censorship as a solution, because the potential for backlash with censorship is far more damaging than trying to work within the system of digital experience and organic community.

 

Dec 16, 2011

3D Virtual Worlds and the Metaverse:

Current Status and Future Possibilities | #SecondLife

 

On December 1st, 2011 there arrived in my email quite astounding news from Dr. Dionisio and Dr. Gilbert that the Association for Computing Machinery had not only accepted our research paper 3D Virtual Worlds and the Metaverse: Current Status and Future Possibilities for publication in their next  journal, but that the reviewers had given the best praise of the paper that the editors had ever seen. As I read through the critiques from each reviewer, I was humbled by their responses.

 

As Dr. Gilbert succinctly put it:

 

After my recent return from the European Summit of the Immersive Education Initiative (I'm still jet lagged as you can see from the early hour I'm writing this!), Dondi sent me the news that ACM Computing Surveys had accepted our paper with relatively minor revisions.  Even more significantly, the impressions of the editors and individual reviewers of the paper were outstanding with all the reviewers suggesting that the work be considered for a best paper award, the main reviewer viewing the work as a  "seminal" contribution to the field, and the Associate Editor calling the first round reviews the best he's ever seen! 

 

What I've also learned is that ACM Computing Surveys is one of the most cited and respected journals in the field of computer science (I had no idea that Dondi had decided to reach so high in submitting the paper). To get accepted into this journal, especially with such high praise, is a great honor and once the article is published our work will be widely read and could have a real impact on the field.  After we submit the revisions in the next few weeks you can crank up all your social media expertise and begin to distribute the article as "in press." 

 

 

LAVA Home of the Future

 

 

 

It’s definitely an honor, and quite humbling to receive that caliber of review from not one but four independent sources across the industry. My understanding of the process was that we should have expected multiple rounds of revisions before being considered for publication, so having it accepted immediately after the first round of peer review is astounding.

 

In the words of the Associate Editor at ACM:

 

These actually are the most positive first-round reviews of a journal article I have seen. If you revise this and include a document that explains how you did, and did not, address comments, I will not need to send this out for further review if it looks satisfactory to me.

 

Let’s put aside the banner waving for a moment, because despite how groundbreaking this all is, it is only a prelude to a bigger picture concerning the Metaverse as I usually provide in articles like this.

 

The Down Low

 

There is a lot of subject matter covered in the paper for ACM Journal, but the title says it all: Current Status and Future Possibilities.

 

Essentially, what we wrote was a comprehensive survey paper looking into the various areas of virtual environments, or immersive environments as some refer to it, and addressed the fundamental challenges and technologies that can further the progress. There is also discussion in the paper concerning what areas are deficient and in need of further research initiative in hopes that those areas will be focused on by some of the countless people reading and citing the paper.

 

Literary influences that led up to the current state of affairs are also outlined, in order to give a proper history of the subject while framing the further understandings as the paper continued on. As one reviewer put it, we presented the most comprehensive paper on the subject that we could short of writing an entire book about the subject.

 

As the message from Dr. Gilbert suggests, I will be making the entire paper available as a PDF when it is sent off to ACM for publication. While there are a lot of topics covered in the paper,  I’ll only be addressing one of them here in this blog.

 

The Take-Away

 

One of the biggest things that can be taken from the paper concerning the future direction of the Metaverse (in my opinion) is the implicit understanding that centralized networking is our Achilles Heel. It always has been, even before we really got into the hardcore 3D Immersive Environments like we see today. All the way back to Lessons from Habitat in 1991, Bandwidth being a scarce resource has always been a concern in something like this, and centralized networking was pegged as a culprit.

 

Yes, we can alleviate that stress by offloading across a datacenter, or buying more powerful servers. But the limitation is still there, and all we end up doing in the process is kicking the can down the road instead of actually trying to solve the problem at the root.

 

Whatever the future holds for immersive environments, it is likely that decentralized networking will be a major part of it in stark contrast to our massive datacenters today and centralized bandwidth.

 

To Boldly Go…

 

There is also a hint toward the data itself, in that the distribution of that data would likely not be the same as how we are accomplishing this task today. I’d like to say that in order for a powerful Metaverse to really evolve, the data has to be agnostic. The best way to understand this is to imagine the replicators in Star Trek and how they work.

 

When the captain walks up to a replicator and says “Tea. Earl Gray. Hot.” the replicator isn’t looking for a cup of tea in its inventory. Nowhere on the ship is there a room with thousands of cups of hot tea on a shelf waiting to be beamed to the replicator. Essentially, there is a holding tank of sorts on the ship which contains the raw molecular “soup” of materials, and the replicators are simply using a digital recipe to pull the components and “replicate” that molecular recipe.

 

 

Janeway and the Replicator

A replicator works almost like magic. Just like the plot to every Star Trek series.

 

 

If we apply this understanding to purely digital files, the same process works. Let’s say that the replicator is a software program on your computer whose sole purpose is to reconstruct digital files based on digital fingerprints (keys) that you give it. Since we’re already talking about a decentralized network as our future, the “holding tank” full of agnostic data to be used for those reconstructions of files should be spread across every user of the network.

 

Why not? We all have a cache folder sitting on our computers from virtual environments and web browsers. What if that cache folder was simply agnostic data and available to the entire network of users in a P2P manner?

 

First off, it would mean that our cache’s are meaningless to human interpretation. Completely random data not representing anything in particular until we give it the fingerprint to reconstruct the file we need. Secondly, it would mean that the data becomes multi-use data.

 

That last point is very important. It means that 1GB of multi-use data represents or contains any file that is 1GB or less. We can say that Photoshop CS5 is under 1GB for the installer, correct? So that multi-use data would contain Photoshop CS5, and anything else that is 1GB or under. The key, so to speak, is that while such a system just about invalidates the premise of current copyright law, making it impossible to prosecute somebody using such a system, it brings up something even more interesting.

 

11 Herbs and Spices

 

It’s not the data that is really copyrightable, but the means by which such data can be arranged in a unique configuration as a representation of a work. What is really copyrightable happens to be the unique fingerprints which tell the system how to reconfigure multi-use data for a specific file output that is no longer multi-use, or in layman terms – it’s the digital recipe.

 

Another interesting side effect to multi-use data is that you’re not storing a 1:1 representation of anything. Since the data is multi-use, that means (for the most part) that you don’t need the storage equivalent for a copy of every single file in the system. All you really need at the very least is enough agnostic data available in order to cover the largest file on the network.

 

If the largest file you have on the network is about 50GB (and I’m being very generous here), then on a multi-use data system, you would need no more than 50GB of data total for everything you ever store into that network. That’s the bare minimum. Yes, at that level it would take longer to reconstruct the data, but it would still work. the more agnostic data you have in storage, the faster that reconstruction goes. Think of it like over unity…

 

Needless to say that a multi-use data structure in combination with decentralized networking would mean that, in example, the entirety of the Second Life asset servers could be housed in about 100 terabytes or less (for redundancy), and the reliability of that system would mean that the odds of anything in your inventory ever disappearing would be so close to nil as to make no odds.

 

And that, is how you solve the root of a problem instead of addressing only the symptoms.

 

 

 

Feb 27, 2007

Dcentralized Content Distribution Network

What would happen if a single web server were suddenly inundated with an abnormally high amount of bandwidth, such that the cost of serving the content far outweighed the acceptable cost per user ratio intended by the publisher? This is known as the "slashdot" effect and can be applied to a number of scenarios across the digital publishing realm.

What inevitably happens is that, a small publisher of content creates a particular amount of media (be it an article, graphic, executable application, etc) that somehow finds massive appeal and favor among millions of people around the world. This in turn opens a virtual floodgate of people trying to access the content online, and thus the servers (of which the publisher is more than likely leasing) are collectively bombarded with content requests.

For a fortified server, such is usually not a problem as bandwidth is appropriated accordingly. But for low bandwidth servers (or say, publishers who cannot afford large bandwidth) we see either that the website in question is obliterated under the weight of the traffic, or that the host charges massive overages to the publisher - of which the publisher many times cannot afford.

Decentralized Content Distribution in this scenario can easily alleviate the major cause of this, while using very little of the network resources individually, but overall serving the content as though it were from a high speed server. Undeniably, 10k/sec from a large amount of servers adds up in the end just as pennies can add up in the large scale of things.

So say your server was being requested of 10k/sec. On your access per month, this amounts to very little and is inconsequential. But now multiply that by 200 servers, and you see that the content can be served at around 2000k/sec or higher (depending on the actual Kb/sec each server is appropriating). So, like a vest of Kevlar, when the bullet (the millions of requests) hit the main server, the Decentralized Network intercepts the bullet and spreads the traffic across a large area in small doses.

For websites, this seems to work amazingly well and, in fact, some websites currently handle their traffic this way (with 25 million users even though the main server remains low on bandwidth usage). The idea here is that I got to thinking about how this could be used in order to solve an age old problem concerning online Virtual Reality.

In one of the earliest written synopsis for an online virtual community (Habitat) Morningstar and Farmer pointed out that in order for a wide scale virtual environment to continually grow the servers of the content could not be centralized. Decentralization was key to the continued advancement and growth of virtual environments. I would like to believe also that this pertains to the "environment server" as well as the content server (many times these two are seperate), though I am currently focusing on a solution for the content end (with the environment server being a consideration well into the future).

When I muse of this, what I am thinking is that while current systems can easily handle tens of thousands of simultaneous users, the price per user begins to see an exponential increase in cost proportional to the amount of users. In essence, we see an exponential increase when the system is centralized.

Taking this into account, even the best planned and programmed system will begin to see an ultimate limit as to how many users can co-exist per centralized server before the costs of that system become prohibitive to it's expansion. And herein lies the point of Decentralized Content Distribution Networks.

While I am unfamiliar with the detailed mechanics of systems such as Second Life or There, I do understand that they take a fundamentally similar approach to the content that is served in their virtual environments. While Second Life effectively "streams" content using a derivative of the RealMedia system (as Phillip Rosedale was one of the creators of the format) I am want to believe that the content still comes from a centralized system of servers. In this much I am certain, though it is quite possible that the Second Life system also groups persons by node in order to offload the distribution of the content. I could be very wrong in the last aspect, but the behavior of the system seems to warrant such node grouping.

In recent months, I have read such articles wherein a self replicating object in the Second Life environment has managed to disable their servers, so in light of this new information I am now leaning toward the idea that they are attempting to run the Second Life environment on a centralized system.

This in itself is fine if a company in this field wishes to establish a user base of no larger than 100,000, but around such time, the cost per user will begin to skyrocket out of control as the numbers increase past this threshold.

In the end, seeing as I am interested in the theoretical applications of virtual environments and it's advancement, I have found that the Active Worlds environment seems to be the most receptive to this line of testing, and have continued toward attempting to find solutions to problems outlined in the synopsis "Lessons Learned From Lucasfilm's Habitat" Morningstar & Farmer, of which happens to be attempting to address the centralization problem.

Over the past few months I have indeed found a very plausible candidate for solving this Decentralization aspect within the Active Worlds environment by routing the Object Path through a server-side P2P network which self replicates and organizes the information from a main server across 260 high bandwidth servers around the world as a specific type of hash file. The more bandwidth that the Object Path requires, the more efficient the CDN will become, even though in beginning to use it I have noticed variable speeds (but not enough testing or time has been given to this in order to gauge it's overall ability).

One thing that I have immediately noticed is that it actually works in the Active Worlds environment, though entering the specific address to begin with in the World Properties Dialog and hitting apply will crash your browser, after you restart the browser you immediately realize that while it crashed your browser the change in address did take effect and the Object Path is being redirected through the CDN, thus beginning the hash replication and self organization across the whole network (server side and not user side).

After some various testing in the virtual world, I have noticed that it has a 99% accuracy when downloading the various files from the distributed network (99% as in that maybe 5 or 6 files will randomly time out while trying to be located and thus not properly loading, though simply exiting and re-entering the world seems to automatically correct this problem).

While I will not disclose exactly how this method is accomplished, I will say that for the most part it works just fine. I would imagine that it would perform much better under much higher stress than a few users entering at random, so this method would be best used for virtual environments where you are expecting a much higher stress on the Object Path (hundreds, thousands or more users). In theory, it would be possible to minimize the overall bandwidth consumption of an entire universe server's set of Object Paths even if it were to be filled to the brink with users (possibly serve every Object Path in a universe from a single cable modem and a home computer with low latency).

So from the theoretical application to the proof of concept, at least the serving of the content part can be decentralized as suggested by Morningstar and Farmer.