Whatever’s going on in our imagination, we tend to think of it as being like ‘movies’ that we ‘watch’, but this description isn’t doing justice to the way that imagery really works in our minds
We imagine a scene. We imagine someone we know saying something we saw them say, or something we expect them to say. Each imagined moment is like a clip from a movie.
So what on earth is wrong with our time-honoured habit of labeling these kinds of imaginary experiences as being just like movies?
‘We run videogames in our heads’
“Comprehension is grounded in perceptual simulations that prepare agents for situated action”
Learning new things triggers a simulation in our heads which gets us ready to act on on what we learn
For us to truly understand anything that we experience well enough to be able to learn from it and therefore be able to act upon it, we need to be inspired to and be able to run a simulation in our mind that prepares us to act upon that understanding.
If an experience doesn’t prompt our mind to initiate a simulation of a ‘projected implication scenario’ then our minds will not have derived sufficient learning from that experience to be able to act upon it or to take it into account when we act
The resulting mental simulation situates us within a virtual context where we can interact with an ‘implied and extrapolated new world of possibilities’ that the newly experienced thing introduces.
In a learning context, we will not find ourselves automatically running that mental simulation if we have not been able to intuit the imperative to mentally simulate the implications of some newly encountered thing.
The relationship between a teaching attempt and a learning experience revolves around this triggering of a simulation.
It is worth considering what might happen if the newly acquired information proves to be clear enough to trigger a simulation, but proves to somehow ‘fail’ as a result of the simulation revealing some inconsistency or some ramification which ‘doesn’t work out’.
This ‘simulation reflex’ does not seem to have been widely explored.
As well as being a useful test for learning, it seems like a helpful resource for UX research.
The answer is that there’s also a whole bunch of ways that the ‘just watching movies in our heads’ description is more than a bit stretched.
The ‘mental movie experience’ turns out to have some features which have a lot less in common with movie-watching than they have with movie-making.
A very brief teardown of those movies in your head
In your head, who’s ‘directing’ those movies? Who writes the script? Is it you? Do you find yourself deciding which scene to watch, what angle to watch it from? Who’s in charge of your ‘viewing’ experience? Is it you? Consciously? Unconsciously? Is it more like TV than movies, so that you can flip channels whenever you want?
Is it more like viewing on-demand, like YouTube, when you can start and stop playing things whenever you want, or is it more like broadcast TV, where you have no influence over ‘what’s on’?
Not being able to answer these questions very definitively myself, I find that I am almost as dissatisfied with (or confused by) the movie-making label as I am with any kind of TV or Movie-watching one.
Maybe, this is because what’s going on in your head is an ‘interactive experience’: a key figure in the ‘proceedings’ that we encounter in our mental conjuring is ourselves.
You may not always be the star, but are you in the cast or crew?
We aren’t necessarily visible ‘in the frame’ in these vignettes and we don’t necessarily seem to have any conscious say in what happens from one moment to the next, but we often seem to find ourselves ‘personally involved in the action’ in some way or other.
Sometimes the other characters in the scene seem to be talking to us, sometimes they can hear what we are saying to them.
Are we the ghost in our own mental movie-making machine?
At other times, we are certain that we can hear what they are thinking or maybe that they can hear what we are thinking, in a way which seems much more concrete and natural than any ‘impressions of mind-reading capability’ would ever be for most of us in real life.
This level of interactivity, combined with our own involvement in the plot or dialogue, disembodied or otherwise, as well as some uncannily-convincing supernatural powers like telepathy and even wish-fulfillment, makes the whole thing seem as if the experience would be much more consistent with being at the controls of a hyper-realistic video game than watching TV, movies or YouTube.
What does a more interactive model of mental simulation mean for UX research?
I think this means that we need to start thinking differently about many of the problems that we try to address when seeking ‘prospective user insight’ in our experience design efforts.
When we try to solve problems whose solutions are wholly dependent upon how the people who are going to use those solutions ‘think about the world’, we need to recognise that if we somehow succeed in getting those ‘prospective solution users’ to imagine a scene where they are using our proposed solutions, they are to all intents and purposes ‘running a simulation of using it in their head’, rather than just doing the equivalent of ‘passively’ watching a movie mentally evoked by our description or depiction of the solution.
If we start to think more deeply about the mental experience that we can summon up in such circumstances, we need to consider the extent to which the whole fantasy can be much more fluid and interactive than our communication materials are (even if those materials do include a YouTube video of a real or simulated usage scenario).
The scenario being mentally simulated by the prospective user may include aspects which are totally unanticipated by us and whose details we may greatly benefit from learning, if we can somehow manage to get them to vividly convey their ‘mentally simulated user experience scenario’ to us.
The video game-like nature of such mental experiences offers the opportunity for us to ‘go on an exploratory journey’ together with the user, as we get them to re-run and share the simulation experience under different mentally evoked conditions and scenarios.
We nonetheless still need to bring objectivity and caution to such exercises, because even if the prospective user is able to provide a rich and hitherto inaccessible source of insights into the projected usage experience, we still need to bear in mind that this feedback is still nothing more than the results of ‘running a simulation’, no matter how compelling the simulated experience might seem to all concerned.
The fact that the platform that this particular simulation software is running on is more powerful than any we currently know how to build, is an opportunity too good to miss, but we still need to balance this benefit with the caveat that, as a consequence, the results as presented may be overpoweringly persuasive and so they probably need to be weighed up and corroborated even more thoroughly than if the simulation had been running on a computer.
