Transformations for Experience

Back Scroll To Media

Transformations for Experience

Sam Hill

10th January 2013

A while back I mentioned our theory-in-progress: that there are two kinds of design intervention that can improve the human experience. The first are designed ‘events’: finite moments in time, with their own contexts, during which things happen. Lots of people work in producing consumable experiential events, even if they don’t necessarily view them this way – certainly performers, game makers and interaction designers do; but also musicians, film-makers, artists, restaurateurs, etc. etc.

The other intervention type, however, is a little bit trickier and much less common. These are transformations, or augmentations – finding constant, passive, sustainable ways of being. How do you squeeze more life out of everyday living? We’ve identified three broad categories of transformation that would allow the collection of more experience value: sensory augmentation, memory augmentation and attitudinal re-evaluation.


1. Sensory Augmentation

Sensory Augmentation is ‘improving’ the way we interpret the world, which could be done in many ways:

Augmenting our existing senses

We could, theoretically, take our existing senses and improve them with the following abilities:

  • Perceiving beyond our current range, e.g. our vision does not include infra-red or ultra-violet light; our hearing capacity is restricted to a narrow band of frequencies)
  • Detecting things from greater distances (e.g. sharpness of vision, smelling blood in the water that originates from far away)
  • Distinguishing subtle differences between similar sensory inputs (e.g. tasting different varieties of grape in wines, or being able to sing pitch-perfectly)
  • Isolating a particular element amongst broad and varied sources (e.g. picking out a particular voice in a bar)
  • Processing input more quickly (e.g. seeing movement at a faster “frame-rate”, as many birds can)
  • Discerning subtle rates of change (in temperature, light, speed)
  • Observing in a broader directional field (e.g. having greater peripheral vision)

Here’s part of a larger Mezzmer info-graphic doing the rounds. It illustrates how awesomely badass the mantis shrimp’s vision is, relative to ours:


Senses seen in other Animals, not analogous to human senses

With the aid of developing tech, we might be able to equip ourselves with entirely new senses, inspired by other organisms in nature, such as:

  • echo-location, such as that used by bats, or dolphins (granted, some people have mastered echo-location too)
  • chemical detection via a vomeronasal organ, like in snakes
  • electroreception, as seen in sharks
  • magnetoception, as seen in birds

Data-centric Augmentation:

Contextual data could aid our perception and navigation of the social, human-constructed world:

  • universal translators and other aids for communication
  • diagrammatic vision – abstract visualisation of intangible things e.g. showing the electric field around an object, or the presence of radiation
  • annotated vision – providing ancillary data about things seen

Non-naturally occuring senses – including the fantastical

  • “x-ray” vision – seeing ‘through’ solid things
  • thermography – perceiving temperature (edit: some snakes have a crude form of this)
  • tele-sensation – tactile sensation from a distance, perhaps through an avatar/ slave-sensor
  • telepathy – non-verbal/ non-physical communication

Gregory McRoberts used an Arduino Lilypad, ultrasonic and infrared sensors to augment his partially-sighted eye to provide distance and temperature data


But… Would Sense Augmentation Really Increase Experience Value?

We’re postulating on the fly, to be honest. It makes sense that if sensory capacity was enhanced, one would get more from life, but we don’t really have any evidence to back this up. So perhaps we should consider it an opportunity for discourse. There are after all couple of considerations…

The first consideration is feasibility. Can we improve our capacity for greater sensation? Perhaps, even with the greatest bionic and genetic development we couldn’t enhance our senses beyond a certain limiting factor. Even if we could, it seems our minds can only process a finite amount of sensory stimulation at once.

The second consideration is: should we seek to augment our senses? They are, after all, a product of our evolution and should (you’d think) be somewhat attuned to our needs – we’ve actually lost some superfluous ancestral sensory abilities, such as a stronger olfactory ability, as recently as the last couple of hundred-thousand years. It may be that not only does further sensory development fail to provide an evolutionary edge, but possessing it could even reduce quality of life.

For example, Gregory McRoberts says that anyone trying to use his eye-patch on a fully-functioning eye suffers from a form of ‘Helmet fire’ – a term coined in aviation, where stress-induced task saturation, exacerbated by helmet HUDs, impedes pilots abilities to function and make decisions.

See also the clip below of ‘binocular soccer’ – even though binoculars are an accepted form of visual augmentation, if they can’t integrate passively and sympathetically with the other demands of our vision (depth perception and peripheral awareness) they also have an impeding effect:


2. Memory Augmentation

Specifically, enhancing experience would require augmenting Autobiographical memory; episodic memory in particular – recollecting times, places, associated emotions, and other contextual knowledge.

Augmenting Human memory would involve affecting our ability to:

  • record memories – encoding experiences exhaustively, with depth and detail
  • retain memories – remembering experiences for longer/ indefinitely
  • recall memories – have access to memories easily, quickly, entirely, accurately

Pragmatically this can be done in part through existing stuff – tools (such as cameras), systems (such as diaries) or techniques (such as mnemonics), but conceivably, it could perhaps be achievable in the future through genetics or neural-interfacing bionics.

Of course, the experiences themselves aren’t enhanced, but the memories of them are more exactly and comprehensively stored – so all memories would retain more experiential value.

Some people already have superior autobiographical memory – the condition is known as Hyperthymesia and is incredibly rare. People possessing the condition can recall every detail of their lives with as much accuracy as if it’d happened moments ago. Once again, however, we need to question if this capability really improves the human experience. The condition isn’t always regarded as a ‘blessing’; some affected experience it as a burden, and many spend a great deal of time dwelling on the past. The condition challenges the traditional notion of what healthy memory is, prompting the attitude “it isn’t just about retaining the significant stuff. Far more important is being able to forget the rest.” [(via Wikipedia) Rubin, D. C., Schrauf, R. W., & Greenberg, D. L. (2003). Belief and recollection of autobiographical memories. Memory and Cognition, 31, 887–901.]


3. Attitudinal Change/ Value Re-assessment

Finally, sense- and memory-augmentation won’t reap much benefit for any individual unless they have a sincere interest in exploiting such capacity to gain more experience value.

Attitudinal change, based on a reassessment of values, is a much less technology-orientated intervention. Instead it is a cognitive shift; a willingness to perceive ones environment more actively, and with a greater attention to detail.

On the one hand, this can be thought of as a learned skill. Sherlock Holmes, for example, is the archetypal ‘observer’ – someone who is ceaselessly, lucidly, taking in the details around him, analysing them and extracting wisdom. On the other hand, there is also a broader philosophical element, or at the very least a set of arguments – statements for why seeking out and making the most of life’s variety is of benefit to us.

To cause a change in attitude and behaviour would require:

  • Learning how to stand back during autobiographical events and and have an absolute, lucid, sensory attentiveness to what’s happening.
  • Being able to objectively reflect on one’s own motivations, decisions, actions and emotional state.
  • Training emotional and intellectual post-analysis (reflection, critiquing)
  • understanding why it is desirable to maximise one’s experiences

People talk about an ability to “live in the now”. Often it is seen as a good thing, though sometimes the phrase is used pejoratively to imply an inability to see the consequences of actions. It is contrasted with both those obsessed with and living inside of their memories, and those who cannot appreciate the here and now because they are constantly looking for the next thing.


Mindfulness‘ is an essential tenet of Buddhism. Considered to be one of the seven factors for achieving spiritual enlightenment, it is a ready-made, tried-and-tested tool kit for gaining a greater appreciation of one’s environment. Mindfulness teaches both the importance of being aware, as well as providing instructions into how to achieve such a state. Unavoidably, it’s is a very trendy concept in the west at the moment, mainly because it’s lessons can also be applied as a form of cognitive therapy, to help temper conditions such as anxiety, depression and stress.

From our perspective, the applicable teachings of Mindfulness are very interesting, and as we continue to investigate them we’re keen to see what can be learned. However, without wanting to diminish it’s spiritual salience, nor the significance of being able to help therapeutically, as experience designers we need to be sure we can (if possible) isolate the processes from the spiritual.


Experiential Research?

‘Medicine’ is an enormous branch of applied science; a collection of inter-related but distinct areas of study, implicitly dedicated (arguments for quality of life to one side) to the goal of prolonging human life, through countering disease, environmental harm or genetic conditions.

Imagine if there was another contrasting branch of science, called Experiential Research. Experiential research might be considered to have the same ultimate purpose as medicine, but with an approach pivoted at 90 degrees. That ultimate goal being to fit more living into a life, but the change in tact being to get more intensity of living into each minute, rather than increasing the number of minutes.

Obviously, the underlying sciences already exist – biochemistry, bionics, cybernetics, genetics, psychology, neurology, human-system interfaces and data visualisation. The concepts of “body-hacking”, or “super-senses” are not new either. But if there was more collaboration across these disciplines, with the aim of creating new transformative interventions, then perhaps we could all reap the benefits of new capabilities and new perspectives.


Experience, Identity and Ideology

Back Scroll To Media

Experience, Identity and Ideology

Sam Hill

29th June 2012

There is an integral relationship between personal experience, identity and ideology. This relationship was touched upon briefly by political philosopher Robert Nozick in the “Experience Machine” thought experiment, which he published as part of his seminal work Anarchy, State and Utopia in the mid-seventies. The relationship was also discussed in more detail about ten years ago by Harvard business academics James Gilmore and Joseph Pine in The Experience Economy.

What makes these two points of view particularly interesting is that they both attempt to place the value of experience within a greater context – spiritually, economically and philosophically. Nozick treats personal experiences, personal identity and the application of personal belief as parallel value sets (intrinsic multism), but does not necessarily assign them relative worth (only stating that experience is not the be-all and end-all). Gilmore and Pine however describe experience and transformation as progressive stepping stones towards the pursuit of something altogether more spiritual – a monistic eternality, religious imperative or transcendent value (did I mention theirs was a business book?).

Both of these arguments go some way to marginalising the value of experience. Nozick is more directly attacking hedonic utilitarianism than what might be called ‘experientialism’, but his arguments still appear (certainly at face value) to be valid by proxy. Gilmore and Pine believe experiences to be important (that is the main purpose of the book) but ultimately only as a means to an end. They don’t consider experiences to be intrinsically valuable.

Nozick’s Experience Machine

Here’s a heavily abridged version of Nozick’s Experience Machine thought experiment. I would highly recommend reading the whole chapter (… and the whole book) but this is all that is immediately relevant – Nozick is discussing the implications of a more-or-less permanent existence within a simulated reality:

“Suppose there were an experience machine [1]  that would give you any experience you desired… Would you plug in? What else can matter to us, other than how our lives feel from the inside?… First, we want to do certain things, and not just have the experience of doing them. In the case of certain experiences, it is only because first we want to do the actions that we want the experiences of doing them or thinking we’ve done them. (But why do we want to do the activities rather than merely to experience them?) A second reason for not plugging in is that we want to be a certain way, to be a certain sort of person. Someone floating in a tank is an indeterminate blob. There is no answer to the question of what a person is like who has long been in the tank. Is he courageous, kind, intelligent, witty, loving? It’s not merely that it’s difficult to tell; there’s no way he is. Plugging into the machine is a kind of suicide… But should it be surprising that what we are is important to us? Why should we be concerned only with how our time is filled, but not with what we are?

…We learn that something matters to us in addition to experience by imagining an experience machine and then realizing that we would not use it. We can continue to imagine a sequence of machines each designed to fill lacks suggested for the earlier machines. For example, since the experience machine doesn’t meet our desire to be a certain way, imagine a transformation machine [2] which transforms us into whatever sort of person we’d like to be (compatible with our staying us). Surely one would not use the transformation machine to become as one would wish, and there-upon plug into the experience machine! So something matters in addition to one’s experiences and what one is like… Is it that we want to make a difference in the world? Consider then the result machine [3], which produces in the world any result you would produce and injects your vector input into any joint activity… What is most disturbing about [these machines] is their living of our lives for us… Perhaps what we desire is to live (an active verb) ourselves, in contact with reality. (And this, machines cannot do for us.)”

From his exploration we can summarise the values that Nozick’s thought-experiment reveals to be apparently important to us:

To live, he infers, is the combined product of many things – reducible to a number of core values. Nozick hints there may be more then he has mentioned here.

Gilmore and Pine’s Transcendence of Experiences to Transformations

Gilmore and Pine write:

“… Remember, in the nascent Transformation Economy, the customer is the product and the transformation is an aid in changing the traits of the individual who buys it… Certainly, competitors can duplicate specific diagnoses, experiences, and follow-through devices, but no one can commoditize the most important aspect of a transformation: the unique relationship formed between the guided and the guide. It is the tie that binds.

An offering of a higher order can supersede lower-echelon relationships. But the only offering that can displace a transformation is yet another transformation – one aimed at another dimension of self, or at the same dimension but from a different world-view. By world-view, we mean a particular way – often religious or philosophical – of interpreting one’s own existence. In the years ahead, we think that companies and their customers will increasingly acknowledge rival world-views – ideologies, if you will – as the legitimate domain of business and as differentiators of competing offerings… Consider the fundamental nature of each offering:

  • Commodities are only raw materials for the goods they make
  • Goods are only physical embodiments of the services they deliver
  • Services are only intangible operations for the experiences they stage
  • Experiences are only memorable events for the transformations they guide

Then reflect on our personal belief that:

  • Transformations are only temporal states for the eternalities they glorify.

All economic offerings do more than effect an exchange of value in the present; they also, implicitly or explicitly, promote a certain world-view. In the full-fledged Transformation Economy, we believe buyers will purchase transformations according to the set of eternal principles the seller seeks to embrace – what together they believe will last.”

Now, when we compare Nozick’s derived values with Gilmore and Pine’s Progression of Economic Value we can see (assuming of course that one can agree firstly that ‘transformations’ equate to a matter of identity and secondly that any legacy we set out to achieve aligns with our life stance) the same three values are presented, but this time in a hierarchy based upon the economic principal of mass customisation. This can be shown as such:

Criticism of these approaches

Nozick, Gilmore and Pine are prolific thinkers, and I massively respect their work. For the most part these are both compelling theories (and it’s interesting to observe the parallels) but in each case I feel their ideas should be scrutinized further – mostly Nozick, who’s hypothetical experiment – brilliant as it is – was built seemingly tangentially on the way to another argument.

Nozick states: “We learn that something matters to us in addition to experience by imagining an experience machine and then realizing that we would not use it”. This seems unduly dismissive. To begin with, people already do embrace fore-runners of the experience machine – the ever-growing computer game market already provides us the capacity for continuous, uninterrupted synthetic experience. Condonable or not, people have dedicated years to their MMO accounts, RTS ladders and Xbox Live scores and at the extreme this has had the predicted detrimental effects to their real-world development and activity.

One might argue then that Nozick is really saying “…we should not use it”. Well, in this case, we’d need to rule out the possibility our reservations are irrational, or poorly founded. Perhaps, for example, the association we have with the word “machine” is too strong; we can’t fathom the idea of a truly authentic digital experience – even with all the caveats in the world – if the uncanny valley has left us irreparably prejudiced.

There seems to be a disparity, for example, between what constitutes an ‘authentic’ orchestrated experience, depending on whether it is positive or negative. An individual recounting the value of a positive digital experience might be scoffed at for buying into something synthetic, but would anyone doubt the sincerity of any trauma derived from an artificial nightmare?

An important clarification should be made of how the ‘experience machine’ really functions. Does any free will still exist within the system? Would free will matter if the system still compensated accordingly to produce the same outcome? Is the individual obliged (unknowingly) into an unwavering destiny – as if tied to the front of a train – or can they exercise even a modicum of choice – like an oarsman travelling along a river? The latter still allows an individual to be something other than an “indeterminate blob”. Perhaps identity is found in choice, and the broader the scope for choice, the better one’s identity is defined.

Who we are and what we do are very difficult things to untangle. In the same way that cinema uses frames to create the illusion of a moving image, consecutive experiences might create the illusion that there is a consistency in our identity. We’re so dependent on the contexts we find ourselves in when deciding what choices to make. But does identity exist in a vacuum?  Are we really so immutable, or is this illusion a consequence of story-telling – the ubiquitous myth that we are like the characters we learn from in narratives – predictable, serialised and defined. Are we all that different from how everyone else would be, if obliged to wear our shoes?

The nature of ‘transformation’ – an effort to change one’s identity – further compounds this abstraction. How can someone will themselves to be different? How can they will themselves in the future to make a different decision to one they’d currently make? What voluntary procedure helps an individual do this?  As Nozick says: “imagine a transformation machine which transforms us into whatever sort of person we’d like to be (compatible with our staying us)”. The idea is nonsensical; the caveat is too open-ended. If identity really is a constant, then to be a different person – by definition – would stop one remaining oneself. The best we can hope for is to change our context.

This is what really happens when individuals use the closest equivalents to transformation machines – they visit therapists to re-frame the perceived consequences of their behaviour; they pay plastic surgeons to make cosmetic differences to their appearances; they take drugs to alter their perceptions and they enter lotteries to change their wealth.

Defining Transformations

It’s useful to understand personal transformation by thinking of the industries that deliver changes to ourselves. Many of these industries are more typically associated with bringing traits of inadequacy up to a median standard, but some are concerned with the improvement of oneself above and beyond what is considered ‘normal’.

We have, for example:

  • Medicine, dentistry and healthcare
  • Personal (physical) trainers and gymnasiums
  • Mental health care – psychiatry, therapy
  • Self-help literature, courses and life coaches
  • NLP and hypnotherapy
  • Skills training and further education – tutors, teachers, mentors
  • Dietary specialists
  • Libraries, wikis and learning/ information resources
  • Beauticians – plastic surgeons, cosmetologists and fashionistas
  • Bionics, mobility devices and sensory augmentation (including eye wear and hearing aids)

(it’s important to note that these are self-initiated transformations. Not included here are the coercive transformations applied to individuals by others, groups, states or organisations – such as advertisement and compulsory education)

When compared with what are considered to be established and accepted experiential industries we notice something quite interesting. For example consider:

  • Performing arts – cinema, theatre, dance, music
  • Sport – played and spectated
  • Literature – poetry and prose
  • Fine arts
  • Events and installations
  • The gaming industries
  • Leisure and tourism
  • Cuisine

What is the difference between these two groups of industry? Moreover, what’s the difference between a diet and a feast? What’s the difference between an hour on a treadmill and an hour at a theme park?

The distinction is not that one is emotionally positive whilst the other is negative, because that is not necessarily important for experiential richness. The distinction is instead that one is stimulating and novel, whist the other is more commonly drudgerous and repetitive. The very purpose of the transformative industries is to assist people in making changes about themselves, and the reason people have not already made these changes independently (and thus demand the ‘service’) is commonly a matter of motivation.

There are a couple of points to conclude with here. First of all, how can a transformation be considered a transcendence of an experience, as Gilmore and Pine describe? How can a calorie controlled diet be the transcendence of the food consumption experience? Secondly, what happens when Nozick’s derived values conflict – i.e. between what we want to do and what we want to be? I might want to be as healthy as an athlete, but I still want to experience sedentary pursuits and eat chocolate cake.