Augmentation, the Metaverse and the Magic Circle

I was deeply offended. It was 2008 and I had been called an augmentationist.

It sounds quaint now. You probably blinked and then asked: "huh?" or "and?".

But back then, it was important what camp you were in.

You could be an immersionist. You believed that computing spaces were evolving to be places. When you logged in, you had permission to leave your physical body and identity behind.  Because these were 'real' places, you believed that they should be governed as such. The rules didn't apply to "users". Instead, people talked about an Avatar Bill of Rights. Your game character had agency.

This was in contrast to the augmentationist. They believed that computing spaces belong in the same category as going to church or the workplace, that they are a social construct (and one of many):

In academic literature, certain online games and services are referred to as “virtual worlds” and compared to cities (Taylor, 2006: 21), countries (Castronova, 2006b) and most frequently, the Earth (e.g. Castronova, 2002; Castronova, 2006a; Nash & Schneyer, 2004, Lastowka & Hunter, 2004). Such language is intended to communicate the scale and complexity of these systems and the activities that take place within them. But the powerful metaphor also affects the conceptual framework from which researchers draw their research design.

In other words: don't call them worlds. Logging in to a 'game' is no different from going to church. It's a place a 'user' goes, where they socialize, where they might present themselves a bit differently than if they were going to work - but it's still them.

I subscribed to much of what Castranov saw in the economies of virtual environments. He argued for the concept of synthetic worlds and pointed to World of Warcraft as an example of how real economic value would be created within the confines of a 'world space'.

In fact, Castranova believed that there would be so much value created in synthetic worlds that that reality would have a hard time competing.

In a follow-up book, he proposed that getting a driver's license would need to be more like a quest in World of Warcraft. The 'real world' wouldn't be able to compete with synthetic worlds for attention and value. We'd need to gamify everything and make 'meat space' a lot more fun.

He proposed that a migration was happening: instead of moving from one country to another, we were moving en masse to synthetic worlds, which would soon have the GDP of small nations. Reality, he said, would need to learn how to compete.

Tom Boellstorff, a noted anthropologist, treated virtual environments as sites of study. He visited the 'natives' much like Margaret Mead visiting Samoa, except that he didn't need to leave his desk to arrive.

I was (and am) a huge fan of Tom's work. He made me understand the power of synthetic worlds not just as a place to 'play' but as sites for cultures that could not exist in any other way.

In 2009, I wrote that:

Boellstorff makes the case that the counterpoint to virtual worlds is not the real world, but the actual. And that this virtuality includes two important things: it is virtual (of course, but he explores this with incredible insight and finesse) meaning that you are never QUITE there... and two, it is grounded in craft, in techne, and that virtual worlds may be a harbinger of a shift from a knowledge or information culture, into a craft-based one….or perhaps a mash-up of the two, what’s now coming to be known as “crafty knowledge”.

The concept of techne was profound for its time. It included the concept that while human culture was informed by our tools, we had entered a new era in which the tools to craft with were resident in the worlds that we created. Our tools now contained tools.

It was also a harbinger of today's creator economy. Because at the time, the idea that you could use digital tools to craft digital artefacts and that those digital objects could be sold was...new....strange...and exciting.

Entire legal conferences were held on the sole topic of how to wrangle this idea of digital assets having value.

An article that came out of a now out-of-print collection was called, if I remember it right, "I am not a hammer". It took the legal argument that just because you call something a hammer in World of Warcraft you can't value it as a hammer: it's actually just code, it's just software, and the rules that apply to software should apply to that hammer.

I disagreed. I argued that we had entered a new era where, in spite the prevalence of case law, the old rules would no longer apply. The lines would blur between physical and digital objects.

If that hammer is used to beat someone up, it might still be considered assault, whether the hammer is real or not. And that context and the meta-governance of digital spaces would play a critical role.

And so it was with this background, after having written around 400,000 words on the 'immersionist' landscape, that someone called me an augmentationist.

And so I was offended.

I owned a digital house, had an avatar and even made some pretty good money in virtual environments.

It was an era where avatars had names, and it was the names that were important, not the gamertags of those "operating the avatar", let alone the 'real names' behind them.

And so how dare someone call me an augmentationists?

12 years later, I realize they were right. But it was because the world(s) themselves had changed. Culture was shifting. And in keeping with this shift I was about to move myself towards work that asked a different question: what happens when the physical world becomes a digital channel?

Kevin Kelly On Identity

Over a decade ago, Kevin Kelly laid out the road ahead:

"A major theme of this present century will be the pursuit of our collective identity. We are on a search for who we are. What does it mean to be a human? Can there be more than one kind of human? In fact, what exactly is a human?

We get to play with answers to these questions online. In Second Life, or in chat rooms, we can chose who we want to be, our gender, our genetics, even our species. Technologies gives us the means to switch genders, inhabit new forms, modify our own bodies.

At the same moment, we have the rise of hyper-realities. These are simulations so complex, convincing, and coherent that they have their own reality force. A fake so good, it is sold and bought as a fabulous fake. A Disneyland so enticing, that it spawns its own “fakes.” There must be something there to fake. Or Photoshopped images so obviously unreal that they have their own reality. Synthetic materials more desirable than natural ones. Originals inferior to their reproductions. Who cares what is real and what is memorex?

These hyper-realities launch questions such as whether a assault in virtual space counts as an actual violent assault or mere virtual assault. How much of our real lives is mental? How much of reality is a consensual hallucination? Where do our minds end and outside begin? What if it — everything outside of us — is all mind?

The faster and greater our lives become mediated — the more time we spend communicating through technology — the more urgent this question of “what is real” becomes. How do we tell the difference, if any, between realities and simulations? How do these redefine humans?"

It turns out, of course, that a few things were true:

• Yes, "we get to play with answers to these questions online". But, at some point the "play" would need to end. We'd need to stop having proxy discussions about identity, ownership, hierarchy, governance, privacy and the rights to creation. We needed to (and still need to) work these things out in the 'real world'.
• The profound implications of simulations, fakes and 'what is real' have just begun.
• There are profound changes which have nothing to do with technology. Those changes are forcing an even deeper reckoning with the question of "how do we define what it means to be human". Whether technology trends towards immersion or augmentation might have profound consequences, but pales in comparison to whether our planet will survive the answering of the question.

Epic, Apple and Why the Augmentationists Won

All of these things are background to what I was thinking when the Apple/Epic battle broke into the open.

My initial reaction was to fall back on the old tropes, to use the old categories:

• Apple is an augmentationist. Its mission is to 'enhance' reality and to create products that we use in our everyday/"real" lives. It's why I argue that Apple will never really launch a "VR headset".
• Epic is an immersionist. They dream of the Metaverse. Which means, by definition, that they need to fight for a universe which is self-contained, with its own economies and currency and systems of identity. The battle over the App Store is a battle for ownership of the entry points to an immersive universe.

My categorization wasn't meant to be definitive. I thought it might be a useful guide to thinking about two radically different visions for our technological futures.

The battle for the App Store has almost nothing to do with commissions. It's a battle for where control will sit as we migrate into ever-more synthetic experiences.

Before we go on, I'd like you to subscribe. It's free and you'll receive updates of new posts by email. I guess it's what I'm asking for in exchange for taking the time to share with you like this - a chance to pick this up again later. If you got this by email already, thank you. Feel free to share!

But the old categories felt....well, old. They felt like yesterday's language.

And I started wondering why.

It felt clear to me that the augmentationists had won. We don't talk anymore about our avatars having cool clothes from Prada or wherever. Instead, we talk about how WE can buy cool digital clothes (and incidentally use them to attire avatars):

Yeah...here we are in 2021 and people are coming up with the same stuff that Castranova did in 2005.

But the language has changed. We're no longer asking whether this is an exodus into virtual spaces. It's now a given. The question, instead, has shifted to share of wallet.

In other words, it's now assumed that we're always logged in. The debate over immersion seems decidedly old school.

Voice, Baby

Back when I was accused of being an augmentationist, online voice chat was still new-ish but starting to grow. It was one of the early markers of a significant cultural shift.

Later, Discord let us voice chat while we played games. It was a further erosion of the boundary between us and the characters we played. It's hard to maintain a willing suspension of disbelief when the woman you're hanging out with suddenly speaks and is, gasp, actually a man.

Twitch took this further: now, it was ME playing a GAME. And the ME in that equation became the focal point.

At one time, this would have led to howls of derision: "if you're just an operator, then you're not a real gamer/player/avatar/citizen".

Now, some of those operators were pulling in 6 figure incomes on Twitch and the personality of the 'operator' mattered a great deal.

Today we have Clubhouse. Philip Rosedale (who founded Second Life) is playing around with spatial audio.

This shift to voice, and then the shift to adding video as an overlay to synthetic worlds, put an end to the immersionist fantasy.

The membrane around the Magic Circle had become very very permeable indeed.

We could suspend our disbelief and wander around Valentine but we could never be sure that our experience wasn't being streamed on Twitch somewhere, and wouldn't know whether our lame gunplay was being laughed at or upvoted on a subreddit somewhere.

The only true way to enter a synthetic world was to play solo. But then, there's not much of an economy or culture in a world of one (notwithstanding Death Stranding).

It's tempting to think that this was some sort of meta world: that Twitch was a world in which we watched 'players' who participated in 'worlds' and everyone was invited to the back chat.

Twitch, in one view, was a sort of fun-house mirror version of Boelstorff's techne. Tools embedded in tools embedded in tools.

But that's too geeky.

The truth is that Twitch and Discord and the increasing fragility of the Magic Circle were emblematic of something far simpler.

Worlds Became Media

Which brings me to the single most significant shift in how we think about "worlds" and why 'immersionists' are hard to find these days.

And it's because of something I didn't entirely anticipate at the time.

I used to produce a show called Metanomics. It was a serious show, with serious people, and talked about the serious impacts that synthetic worlds were going to have. (Episodes are still floating around on iTunes somewhere).

Sometimes I feel like the topics would make a good Clubhouse chat. We had a good one about The Customizable Body, for example:

We'd have people on who talked about machinima and the birth of a new art form (now called 'virtual production', which gives you a hint at how quickly the language and culture can shift).

The breakthrough we thought we'd made was that media could be produced inside the environment. Which was definitely profound. It was an extension of Boellstorff's concept of techne.  

What I didn't anticipate was that the media around a game would be of an exponential size or have such incredible influence.

Game companies became media companies. The media ecosystem around a game became more important and more influential than the 'techne' that could be created inside a game.

The early machinima filmed in Grand Theft Auto was far less important than someone live streaming on Twitch or a 2 hour Let's Play.

Media had consumed synthetic worlds.

Scan Everything

But this still doesn't entirely encompass how profound the shift has been.

For 6 years I blogged about the Internet of Things. When I first started, I had a simple premise: that our phones and other devices could only very roughly detect the world around them. And that IOT devices gave an early hint at what it would be like if our devices could "see".

I proposed that the world would become a new channel.

I think I saw this as a separate argument to that one about immersive vs augmented worlds. Maybe I was thinking that the world itself would become immersive (and yes, it still will be).

But what the technology of the past few years has shown is something much broader.

It's hard to detect it in a Snapchat lens or a LiDAR scan posted to SketchFab.

And it's the realization that these were all dress rehearsals.

Whether a synthetic world, hanging out in a VR space, scanning your sneakers, sending a Snapchat AR-enhanced selfie or consuming a spatial audio experience, they're all early indicators that we are approaching escape velocity in digitizing the human experience.

Synthetic worlds may have seemed, on the surface, like a shift towards creating new realities: synthetic or virtual alternatives to 'actual' life.  But this appearance disguised the fact that these were the first rough attempts at digitizing the human experience itself.

People were falling in love, their avatars were making millions in real estate, and we were active participants in non-physical realities - and our brains sometimes didn't bother to differentiate them as "not real".

And in doing so we discovered that everything is or can be media.

The important question, therefore, wasn't "am I an augmentationist or immersionist?" The question was:

If we digitize everything, is there anything that will not be media?

Everything Is Digital

Stories are culture. Stories are also networks. When everything is digital, everything can become media.

There is no boundary between your body and a digital self:

The MetaHuman Creator isn't a 'character creator'. It's the first step in creating a mirror world of selves.

The AR Cloud isn't just a mirror world. Digital twins will, first, help us perform simulations of...everything. But later, they will be living, breathing ecosystems which take signals from the 'actual' world. They will let us create real-time media of distant places, with real-time traffic patterns without ever leaving 'the cloud' (or edge).

NFTs are an attempt to nail jello to the wall: they try to codify a transactional value for something which is infinitely replicable. But they serve a more profound purpose because they allow us to leave a mark, to say "this particular story, this particular piece of media was seen, and consumed, and then valued".

NFTs will disappoint a lot of people in the short term. But in the long term they are the codification of our reactions to media. They create a (theoretically) immutable register of which stories/which media matter.

On the one hand, it's all slightly dystopian. We will all be participating in the Truman Show. Our life won't just be streamed from Fortnite. We'll inhabit a world where we'll look through AR glasses, and the world we see will be populated with Pokemon and Star Wars characters, whales and MetaHumans.

The genie is hard to put back in the bottle. Like the Truman show, nothing will be hidden from the all-seeing stream.

The physical world will broadcast back into mirror worlds, and mirror worlds will stream out to the glasses we wear while drinking (hopefully) real cups of coffee.

On the other hand it might be empowering.

Yes, it will be hard to not be logged in. Siri is listening. And so is your robot dog, your eyewear or your tokenized running shoes.

The good news is that the stories themselves will still matter. That stories are the containers for our agency and values, our aspirations and identity, and how we share culture with each other.

So long as still have some degree of agency in the stories we tell, perhaps the expansion of the canvas on which we create and consume media, will still allow us to answer the questions Kevin Kelly asked:

What does it mean to be a human? Can there be more than one kind of human? In fact, what exactly is a human?

In an interview I did with Tom Boellstorff in 2008 he said:

In the western tradition, the origin myth for knowledge is the tree of the knowledge of good and evil, and the Christian tradition is the best known example. And the back of my Apple computer has a picture of that, right, the apple with a bite out of it. But in the old Greek mythology, the origin of craft is from Prometheus stealing fire from the gods. And, in the original Greek, what he actually steals isn’t just fire, but the ability to use fire, which they call techne, this ability to craft things.

So maybe, in the end, it's really the immersionists who won...because software truly has eaten the world. Once given the gift of fire, and the gift of crafting things, it was always a given that the stories we told would go large enough to encompass....everything.

We are swimming in the sea this digitization of everything. The process is happening with alarming speed.

We were given the ability to use fire. To craft. To create. To burn our stories, to etch them into everything.

Now. We find out what only the gods, until now, have known.

Thanks for making it all the way through. Please do reach out. I'd love to hear all of the ways you disagree, or all the ways we can have a conversation together.

Thank you,

Doug.