Forgotten your password?
David Parry / PA Wire

Do you see what I’m saying? The link between visual culture and empathy

Debbi Evans
Visual expressions of emotion online both amplify and simplify our emotional literacy - communication barriers are lowered, but is there a risk that we’re just saying less to more people?

Last month, it was revealed that emoji (picture messaging) is the fastest growing language in the UK, fuelled by the global adoption of smartphones and instant messaging services like What’s App. While it can sometimes feel hard to keep up – or perhaps precisely because of this – it’s clear that pictures > words on the internet. At a broader level, a tacit understanding of ‘pictures or it didn’t happen’ has led to frantic biographical documentation – with visual social networks like Instagram, Pinterest and Tumblr gladly springing up to accommodate. There have been no fewer than three emoji-only social networks – emojicate, and Steven, which is no longer available. This is a shame as it sounds the most interesting, creating an ‘ambient awareness’ of what you’re doing by tracking your activity or location and logging it in picture format (a cup of coffee if it knows you’re in a coffee shop, or a briefcase if you’re in the office). In theory, this allows your connections to have a more realistic sense of your life activities without you having to tell them, or them having to ask.

According to Swyft Media, around 6 billion smileys are sent around the world every day. There are 300 million emojis shared daily on Facebook. Businesses are taking notice: Mentos and Coca Cola have released their own branded sets – in the case of Mentos, accompanied by an expensive campaign that sets out the different personality of each ‘ementicon’ (*shudder*) in an attempt to spark some affinity with their target audience. This is a way of infiltrating natural, everyday communication on the brand’s part, but it also demonstrates just how much the use of visual messaging has skyrocketed.

Tiffany Watt-Smith, who researches and has written a forthcoming book about the history of emotions, concedes that there is a long history of visually representing emotional states. Even the earliest human language started out in image form, first as pictograms – literal representations of objects – before evolving into ideograms, which communicated more abstract contexts. Their online evolution seems to be tracing a similar path. The first emoticon – a simple smiley – was used as a cue for how the message should be interpreted (not seriously) rather than being message itself. Today, we’ve got animated emojis to illustrate an increasingly diverse set of situations and even translated versions of literary classics, such as Emoji Dick, which raised funding on Kickstarter in 2013.

Controlled communication

There is no set of guidelines for emoji use, and they have no official names beyond those granted by their users. The language is still evolving – in spite of attempts by the Unicode Consortium, who control which emoticons go into our standard smartphone keyboards, to impose some consistency in their use. Google have patented a text-to-branded-emoji translator which, if successful, would introduce a worrying bias into everyday conversation. Emoticons are also being taken seriously at the highest levels of the law. Sally Bercow’s *innocent face* tweet about Lord McAlpine’s sexual proclivities led to a landmark ruling that emoticons and other symbolic expressions of imagery could be as defamatory as words on their own.

But Watt-Smith doesn’t understand their appeal. “I’ve never used an emoticon and I don’t imagine myself ever using them. To me they seem incredibly reductive and limiting. It seems strange that people would find them a useful tool for communication.” I know what she means – or I did when I started researching this piece. For a start, you need to know where to find the images you’re looking for before you can type them into a message, which takes away from the fluency of communication altogether. And interpreting other people’s is an even bigger challenge – one missed word or symbol can change the received meaning entirely. You couldn’t (yet) have a reasoned debate in emoji. Burger lips building. What?

But more experienced emoji users have told me that this does get easier with practice. Like learning a language. And the ability to decipher a message is improved by understanding the sender – their sense of humour, their communication style, their personality. What’s implied is as important as what’s visible. Addie Wagenknecht, co-creator of the Emojinal coding language, says that emojis’ limitations may well be where their power lies. Perhaps this is because their lack of precision enables them to do what one journalist called “emotional heavy lifting” in a difficult online exchange. Even if you’re not sure what someone is trying to say to you, a picture can give you a sense of how they’re feeling about it and make you aware that they’re trying to forge some kind of connection through the wires and tubes.

Show not tell

As we’ve seen, people who are actively engaged in social media – and young people in particular – are constantly aware of their audience and their role as entertainers. Images leave much unsaid and open to interpretation, so their meaning and intention can be defended in line with audience feedback and the threat of social shame. They’re also useful in projecting an aspirational persona, again carefully curated. From one (less flattering) angle, however, this could be seen as a lack of clearly defined personality. With the broadest imaginable pool of influences at their fingertips, people can be everything at once, and this has made them less of any one thing.

Chris Christodoulou, a media theory and music professor at Westminster who has extensively studied various musical subcultures, has seen this non-commital, pick ’n’ mix approach to identity play reflected in his students’ choice of clothing. Whether it’s a Kid’n’Play haircut blended with grungey trousers, “there’s no distinction between subcultures anymore.” This isn’t surprising when you consider the huge, ahistorical pool of imagery they’re bombarded with online – retro gifs, pins, picture memes, Facebook stickers, vintage-style Instagram shots. They can dip into and play with it without needing to commit to an opinion or statement about what any of it means.

Chris has also noticed, anecdotally, that some of the most wildly dressed on campus have the least clearly defined personalities when you speak to them; conversely, I wonder whether the blank canvas of ‘normcore’ clothing is a direct reaction against this. The more boring you look, the theory goes, the more interesting you are. There’s a parallel to this online, summed up in the reference to having “all the feels”. Writer Katy Waldman describes the emergence of an emotion economy – an OTT spectacle of sentiment influenced by reality TV culture. Being seen to care or feel deeply about something adds to your personal brand value. Hence the outpouring of grief whenever a well-respected celebrity or activist dies – are you really feeling those feels, or just needing to express something? There’s an inverse correlation between actual emotional sentiment and its expression. ‘Feels’ are a way of actually distancing yourself from emotion online. In a lot of cases, the more you say you feel, the less real emotion you’re likely to be experiencing.

Cultural difference vs universal meaning

It will come as no surprise to anyone that East Asian countries dominate emoji use, partly because they’ve had them for longer and have therefore got more adept and creative at using them. Even simpler emoticons have more creative iterations, such as the ‘angry table flip’ using Kanji characters. A quantitative, big data analysis of emoticon use in 2014 found that more individualistic cultures favoured horizontal and mouth-oriented emoticons, whereas collectivist cultures tend to use vertical and eye-oriented smileys. An interesting nugget-in-a-vacuum, this, as it doesn’t get us any closer towards understanding empathy and visual expression.

Regardless of any cultural differences or nuances in communication – Spanish-speaking Americans are the heaviest users of the sad face, according to keyboard app Swift Key’s recent analysis – there are roughly two main camps, positive and negative. Over 70% of messages in were positive, perhaps unsurprising when you consider the preoccupation with appearing a certain way to your audience. The only work-related study I managed to find, of remote customer service and telesales workers in Scandinavia, found no instances of the frowning face at all in messages exchanged, a fact the researchers attribute to wanting to demonstrate a positive attitude to co-workers and management.

The universal meaning of the sad and happy face has made them powerfully persuasive when designing for certain behaviours, such as more efficient energy use around the home. More recently, Cambridge University researchers found that putting emoticons on food labels yielded stronger effects on perceptions of taste and healthiness than the colour-coded labels currently being debated by the EC. I refer you again at this point to Google’s branded emoji-translator patent, which gets more disturbing given the potential impact on purchasing decisions.

Empathy and ambient attention

But while it creates a positive feedback loop in a ‘nudge’ situation, choosing between two binary emotions presents challenges for more nuanced reflection or communication. Perhaps we just don’t yet have the words to fit the varied and complex emotional responses represented by the hundreds of available Facebook stickers. And that’s before you include custom emoji; services like Imoji allow you to turn any image on your phone into a custom sticker.

Any erosion in language barriers will need to contend with the incessant drive for individualism as self-defining groups of individuals carve out private spaces of secret meaning. Reaction gifs are an indicator of this – the Jean Luc Picard facepalm clip both communicates a well-known expression for one broad audience yet carries within it a second layer of knowing for Star Trek fans.

But perhaps the most significant aspect of the shift to pictorial communication is that it offers up potential for exploring what Amber Case calls ‘calm technology’. We process images much faster than text (one commonly quoted stat reckons 60,000 times faster), perhaps we don’t really need to focus on them to understand them. They could be background notifications, at the periphery of our vision but no less powerful, as we saw with the now-defunct Steven app and its quiet but persistent emoji life logging.

In an age of chaos and constant noise, there’s value in gentle unobtrusive communication – and visual languages could be one possible avenue for this. As Jenna Wortham writes in praise of her favourite emoji, picture messages offer “a way to be present when there’s nothing to say at all”.

Photo credit: David Parry / PA Wire

Any erosion in language barriers will need to contend with the incessant drive for individualism as self-defining groups of individuals carve out private spaces of secret meaning



To continue reading...
It's free and only takes a moment. You'll get:
  • - Access to libertine content
  • - The weekly newsletter
  • - Alerts about events
  • - Competitions and offers (but we won't spam you, we promise)
Log in
Email address *
Password *
Remember me (see our Cookies Policy)
Forgotten your password? Privacy policy and Terms & Conditions Log in
Forgotten your password?
Enter the email address you joined up with and we'll send you an email with a password reset link.
Email address*