At the peak of the Middle Ages, the Mallorcan monk Ramon Lull wrote a small treatise called The book of the order of chivalry, which became a cornerstone of the genre. In Medieval society, the deal was that peasants farmed the land and left part of their income to those of noble birth – the knights – in exchange for protection. Why then, Llull wrote, was it so important that the knights were instructed about moral virtues? It was not that they were better people. But with their horse and their armour, they were people of bigger potential. A bad knight had a worse impact than a bad peasant. A good knight had a better impact than a good peasant. For this reason, it was more important for everyone to make sure knights became good knights, and demanded higher standards of them than of ordinary men. 

I was recently reminded of that rationale when thinking about the rising potential of modern mankind through the use of technology. Like the knight with horse and armour, in today’s world we command a rising arsenal of objects that increase our impact – Sigmund Freud called us “a prostetic God”. We have immense strength with the use of robots and motors. We can fly with the use of a plane. We can be omnipresent with the use of telecommunication.

But we have more in common with the metaphoric knight than that. The stronger and more protective a knight’s armour became over the centuries, the more limited and partial was their perception. Their field of vision was a small slot in the helmet, all the while the helmet was bumping up and down with the galloping horse. The armour was clanking so loud it was impossible to hear what was going on behind you in the melee. So while the reinforced armour made the knight’s thrust more powerful, it certainly rendered it less precise. Only that did not cool down most knight’s impression of being all-powerful, of being able to act, to compensate any lack of understanding with a healthy dose of testosterone.

I think this pretty well sums up how many of us feel about using modern technology to face complex challenges. 

Today’s technology being more subtle, more digital, more immaterial, the metaphor shifts from the armoured knight to the Sorcerer’s apprentice – maybe taking the healthy dose of testosterone along he way. We tend to overlook consequences of our own actions. Partly because at the beginning, they are practically invisible. We are confronted with a huge amount of information, and there are virtually no relevance filters. Partly, because we tend to have a too narrow view of our playing field. We believe we can disregard what is happening in the wider context. One typical area where this is happening is the interface between technology, business, and politics. 

The Sorcerer’s Apprentice from Disney’s Fantasia

One of the big changes brought about by digitalisation is that it fundamentally increases our range: easy access to more information, or easy access to a large audience. Businesses provide this access, and build models about how to monetize on it. But what also happens is that changes in access may affect changes in selection, or changes in visibility, and as a consequence affect what information does in our lives. 

In 2015, in the race to attend its goal of one billion hours a day of viewer engagement, Youtube changed the algorithm that selected videos for recommendation (the ones that appear on the list on your screen, and those that appear on the main screen once you have finished watching a video). The recommendations have been Youtube’s most successful instrument to keep user’s eyes on the screen, maybe longer than they originally intended – the main commercial objective. But with the old algorithm, those videos that were seen most often, were recommended most often. Over time, recommendations would converge on the mainstream, and be considered a bit boring by many viewers. The new algorithm changed that. It still showed many of the most viewed videos in the context of the one you were watching, but it also showed some of the more marginal ones, maybe less closely related to your topic, but most importantly watched by fewer people. Commercially, the change was a success. By October 2016, Youtube had reached its goal of a billion hours a day. But what happened to the content that people were watching? Well, let’s ask ourselves the simple question: what kind of content, in a given area, is viewed less than the mainstream? Probably the more boring things, and the more extreme. If, amongst the typical mainstream recommendations, you get to see thumbnails of some boring videos, and of some extreme, which of the two are more likely to be clicked? The extreme – for two reasons. First, you see the title and a picture, you don’t see the video yet. And extreme videos tend to have catchy titles. Second, the psychological law that describes what catches our interest, and makes us react and engage: emotions work better than rational, cognitive stimuli. And amongst the emotions, negative ones such as insolence and hatred work better than positive ones such as curiosity and joy. Therefore, in the wake of Youtube’s change of algorithm, obscure youtubers with extreme, polarising standpoints, hate speech (as long as it was just within the rules), conspiracy theories and wild accusations, made the career of their lives. Many, who before were on Youtube as a side job, could now make a living off it. Youtube, as an enabler, contributed to the division and polarisation of our society. Was it their intention? Not at all. They were just pursueing the innocuous task of increasing viewer engagement.