So there I was with an Oculus DK2 on my face in a hotel room that was just off of Bourbon Street. My wife was10 feet away, sleeping deeply in the hotel room’s ginormous bed. I could hear drunken bleats and bellows between an ‘idiocy’ of bros and a ‘screech’ of woo-girls as they passed one another on the street below. (BTW- I’m using these words the way one addresses a ‘murder’ of crows or a ‘crash’ of rhinos). It’s early January of 2015 at about 1 AM, and I finally have the Windows 10 partition on my MacBook Pro playing nicely with the Oculus drivers. I plug in my Wacom pen tablet and boot up an alpha version of TiltBrush (pre-Google). That was the first time I ventured into a ‘mindspace.’
Where was your first experience (with haptics) inside a virtual space?
I can draw a bit, and even in the awkward early days of TiltBrush + Wacom tablets, I realized that most of the tricks I thought I knew about drawing were now moot.
Overlapping objects in the picture plane to create a sense of perspective? Nope — that doesn’t work, and isn’t needed anymore!
Working on the lightness or darkness of colors to indicate form? Throw that one out too but instead move the lights around the space.
In Tiltbrush, one is not even drawing on a flat and static plane anymore, but the creative process is something akin to forming in space. Looking back, what really baked my cookie was the fact that I could change my scale and sit on the virtual lines as though they were highways. This dynamism in scale and the affordance of perspective changes was utterly new and was inkling into what I am calling Mindspace.
N. Katherine Hayles’ seminal essay Flesh and Metal: Reconfiguring the mindbody in the Virtual Environment considers the concept of a Mindbody. Loosely, the mindbody is the dynamically shifting relationship between our corporeal body and our sense of being ‘inside’ our body i.e. embodiment. The Hayles essay uses this idea and extends this concept into Virtual Reality. For readers who study creativity in new media, this essay is a treasure trove- here’s a link to the reading.
Our mindbody seems to thrive in Virtual Reality, perhaps because this ‘figure’ exists in a new kind of ‘ground.’ This mindspace is a new place to form behaviors around data which will give us new relationships to information. In the future, I anticipate novel approaches to UX design due to these new possibilities. Mindbody and mindspace experiences already occur in Virtual Reality. One example is Oculus’ First Contact.
This experience is all about training through neural reassociation. In this novel tutorial, new inhabitants in VR re-wire the signals from their minds to their body’s hands (which are now in haptics such as Touch Controllers) to create new neural pathways. This is achieved by small tasks such as catching butterflies or shooting a zapper at moving targets. These neural reassociations are aimed at the mindbody, and most leave this game with newly mapped brains.
New neural pathways exist for the mindbody and shift from platform to platform e.g. HTC Vive to Window’s Mixed Reality platforms, and game to game e.g. First Contact to Superhot VR. (Don’t even get me started on Beat Saber.) Once the initial pathways are set, it seems that the plasticity of our brains occurs in the form of indexing new experiences and rules within each mindspace(s). Simply put, the constraints our minds and body are familiar with no longer apply in virtual reality because we can program and learn in new mindspaces.
The rules of our corporeal reality such as space and time can be programmed in virtual space.
In virtual mindspaces, we have the ability to ‘play’ with time — slowing it down, pausing it, speeding it up… again, something not possible in real life. Time is funny and we make it up. Quite literally! Just about everything that is ‘common sense’ about time is a technological invention created to promote humanity’s growth. Seriously, read Carlo Rovelli’s The Order of Time to find out just how deep that particular rabbit hole goes. Here’s Rovelli speaking about time. 🤯
In First Contact, one of the disks the Johnny-5/Wall-e/Virtual Boy robot hands you is a hypercube, aka a tesseract. A hypercube is a 4-Dimensional dynamic object and these can’t fully occur in 3-Dimensional space. When the cube appears, all of the objects in the space float around a bit. When the hypercube appears, the programmer Alexander Dracott hints at a profound capability of a mindspace: the fundamental rules of reality no longer apply, and the implications of this new option are staggering.
My career allows me to travel, and I find it fascinating that my mind knows where “I AM” in a city on the other side of the world — even one that I have not visited in 18 years. I suspect this mapping to a location will be similar for our mindbody. There is research that argues the portions of the brain responsible for spatial mapping go haywire in VR. As with the haptic neural reassociations, perhaps this research shows the initial synaptic response of the neural pathways being established for the mindbody in a mindspace.
Anecdotally, after enduring longer durations in virtual reality environments, my own mindbody eventually ‘re-maps’ to each new mindspace I find myself within. A humorous experience that quite literally ‘plays’ with the mindbody’s maps is Virtual Virtual Reality. One of the features of this game is to ‘put on’ a virtual labor access point (a virtual head mounted display) while inside VR. During the gameplay, there is a significant ‘lurch’ that occurs when the mindbody that is inhabiting virtual spaces is abruptly ripped of the virtual-virtual space. It’s a trip.
What will happen to our minds when we start interacting with programmed information at higher dimensions regularly? At Imaginary.org one group of mathematicians/developers is already playing with non-Euclidian Virtual Reality environments! Check out their research here. If you’re fancy, here’s a webXR experience.
Is it possible that neural reassociation around time and higher-dimensional space will occur in virtual spaces?
Several years after the initial developments of my mindbody in that New Orleans hotel room, I am still struck with awe when I enter VR. I wonder what initial changes happened in my brain? I wonder what other, more subtle, corridors of our minds, bodies, and our collective mindbody will change because of the new behaviors in mindspace.
