Reflections on “Meaning”

Existential “meaning” is partly the telling of a story featuring ourselves that is available and consistent with our higher-level representations of the world. It is not, generally, rational; it is more a narrative correlated with a feeling of “selfness” and “correctness”.

For example, think of how you feel when you hear your own internal voice as opposed to hearing another person speak. You feel that the internal voice is somehow “you”. This is not a rational thought, indeed the language of rational thought may be seen, in part, to *be* the internal voice. This feeling breaks down in certain brain diseases, such as schizophrenia. With these diseases, the “me” feeling is lost or broken, and hence the internal voice of “you” becomes an auditory hallucination, a voice of “them”.

Concentrate on this feeling of “you” for a moment, try to explore what it feels like.

Now think about a feeling of “correctness”. This can also be seen as a feeling of “truthiness”. For example, try to concentrate on how the feeling of “1 + 1 = 2”, differs from the feeling of “1 + 1 = 5”. The latter invokes a feeling of uneasiness, an itching to correct. It has tones of unpleasantness. It induces a slight anxiety, a feeling that action is needed. The former invokes a feeling of contentment, that no action is required; it may be contemplated for an extended period without unease of additional thought. It’s a similar feeling to artistic “beauty”, the way we can contemplate a great painting or a landscape.

Both feelings may arise from a common mechanism in the cingulate cortex, a medial layer of cortex that sits between older brain structures such as the thalamus and the higher cortical layers. Indeed, certain forms of schizophrenia have been traced back to this structure. The cingulate cortex may be considered to be the emotional gateway between complex neural representations in the upper cortex and structures that manage low-level sensory input and co-ordinate physiology response. “Sensory input” in this context also includes “gut feeling”, sensory input from internal viscera. Work, such as that performed by Antonio Damasio, shows that this input is important for the embodied feeling of self, e.g. “self” may in part be a representation formed from signals from these viscera. In a not dissimilar manner, “correctness” may be based on a representation of error or inconsistency between a sequence of activated cortical representations. At a very naive level this could be built from familiarity, e.g. at a statistical level, does this sequence match previously activated sequences? Over a human life this is a “big data” exercise.

So, back to “meaning”. Our higher-level cortical systems, e.g. the frontal lobes, create narratives as patterned sequences of goal-orientated social behaviour, which may be expressed in various media (stories, plays, comics, dance, songs, poems etc). For “meaning” to be present, we are looking for strong positive correlations between these narratives and the emotional representations of “self” and “correctness”. What form could these correlations take?

First, let’s look at “goal-orientated behaviour”. The frontal lobes build representations of sequences of other representations. These sequences can represent “situation, action, outcome”. There is some overlap with the methods of reinforcement learning. Over time we learn the patterns that these sequences tend to match (google Kurt Vonnegut’s story graphs). The frontal lobes are powerful as they can stack representations over one to seven layers of cortex, allowing for increasing abstraction. Narratives are thus formed from hierarchical sequences of sequences. (There may also be a bottom-up contributions from lower brain structures such as the basal ganglia, which represents more explicit cause-effect pairings without abstract, e.g. “button tap: cocaine”.)

Second let’s look at activities that are widely reported to be “meaningful”, and those that are not. “Meaningful” activities tend to be pro-social. For example, imagine you are an artist on your deathbed; what feels more meaningful, that you produced a work of art seen by no one or that you produced a work of art seen by millions? I’d hazard that the second scenario provides greater “meaning”. We need a sense that we have affected others in a positive manner. Similarly, does “1+1=2” feel “meaningful”? Do your tax returns feel “meaningful”? Does the furniture in a new home feel “meaningful”? I’d hazard “no”. These things do not elicit a strong emotional reaction. The furniture example is a good one; the furniture in your family home may come to have “meaning”, but only because it forms the background of your social memories. We are social animals, like parrots, dolphins or baboons, and so the social realm forms a bedrock to our emotional states.

For correlations to stick in the brain we need two things: 1) for correlations to be present in the outside world (or at least some situations that form a sensory base to those correlations); and 2) for us to regularly experience these external situations. Here “regularly” means at a daily or at least weekly.

Religions have long been aware of these aspects, indeed we often define “religion” as a structured practice built on a common mythological framework. It is widely reported that it is not possible to feel “faith” without practice. In Islam this is explicit, “Islam” means to submit or surrender; you have to practice to believe. The structure of religion provides the regular experience of 2): daily prayers, weekly worship, and annual festivals.

To provide correlations between feelings of “self” and “correctness” and particular narratives, we need to experience them all collectively. It is important that the narratives are at least analogous to our daily experience. If they are not, we cannot experience them as being “correct” or “true”. All of this also needs to take place below a level of conscious awareness.

Again, we can learn a lot from religion. Rituals light up the brain of those experiencing them. You are the one experiencing the ritual, you are taking in heavy sensory stimulation, and are performing actions within the world. Rituals often involve singing, collective and stylised movement, repetition of motifs. These activate common neural representations each time the ritual is performed. The connections that are formed fuse the self and the experience.

The last part of the puzzle involves fusing the narrative and the experience. The self is thus fused with the narrative via the repeated experience.

In religion, rituals tell a story. The Eucharist is rooted in the story of last summer, Passover the liberation of the Israelites, and Ramadan commemorates the receipt of the Quran. It is important that the faithful act in a manner that is consistent with the story. Although many stories are based on an echo of history, historical fact is not important. More important is that the story, or an abstraction of the story, mirrors experience outside of the ritual. For example, many religious stories are based on familial relations, which most can instantly relate to. Many religious stories acknowledge suffering and struggle, as well as moments of joy and exhilaration, which again people regularly feel in their daily lives. The stories are also dynamic, they emerge from history through retelling, emphasis, interpretation. Like our own memories they are recreated every time they are retold. A key role of clergy is to draw parallels between these stories and our daily tribulations.

Having considered these points, we can see why the rational secularism of modernity often leaves people cold and lacking meaning. Science is not a vehicle for creating human meaning. Indeed, I would go as far to say that the factors that make science successful move us in a direction away from meaning. The workaday stories of science need to be sterile for science to work; they need to be objective, unbiased and unemotional. They then provide us with predictive power. But predictive power is not emotional resonance. The predictive stories of science, equations and theories, are not human-centric in a way that matches what we feel. If they were, we would all be reading scientific papers as opposed to watching Netflix.

If people lack meaning in their lives, they quickly fall into nihilism and despair. The challenge of Western post-modernism, having largely ditched religion, is thus to fill the void and create something we can use in our daily lives. Science and engineering could provide the tools and understanding, but they will not provide the solution themselves.


Reflections on Rising Nationalism

Toxic sentiment in society doesn’t arise from a vacuum. History shows that it tends to follow a perceived threat to a way of live, a reaction to change. You cannot remove the toxicity without addressing the underlying issues. But what are these?

***Caveat*** This is just my opinion based on what I have experienced and read. It may be wrong in whole or part, there may be errors or omissions, or you just may plain disagree. Also auto type on the iPad leads to annoying typos that appear stupid. ***


Many in the UK are experiencing a rise in racism, nationalism and anti-elite prejudice. This by itself is not unusual; such feelings often come in waves in reaction to rapid changes in society. The turn of the 19th century saw a similar expression. You just have to read the novels of Joseph Conrad to feel the tension in the air. The change in sentiment at the moment though seems particularly disorienting, it was only six years ago that Britain celebrated its history and diversity at the Olympic opening ceremony.

So what has changed?

I think the changes can be broken down into two sets: more stable long-term trends and shorter term issues. You can think of this like modulating a radio wave, we have the carrier wave slowly applying change and a signal that flits up and down upon the carrier wave.

The long-terms trends can be split into: globalisation, perceived immigration, aging populations, education and liberalism.

The short-term trends can be split into: the “Great Recession”, wealthier pensioners, the rise of social media, the decline of newspapers and a rise in alternate political structures.

What are the long term changes?


More precisely, an increased ease in physical transport and a greatly-increased ease in global communication.

The container ship allows goods manufactured on one side of the world to be quickly shipped in days or weeks to another side. Modern logistics mean I can order a product online that is built thousands of miles away and have it delivered to my door within 24 hours. It is common for those in Britain to fly to America or Thailand, and for you to see Asian tourists in provincial British university towns. Fifty years ago you could not fly around the world in a day.

I saw this growing up. My family worked making shoes. In the 1980s manufacturing moved from the UK to Portugal. In the 1990s from Portugal to Vietnam and Thailand, then to China. In the 2000s it flitted between East Asian countries following labour costs, skills and infrastructure. No one makes shoes in Somerset anymore.

Physical transport, giving way to phone and fax, and then the Internet and email, means that information can now flow around the world in milliseconds. This makes it easier to coordinate global logistic chains and also means that cultures are merging and converging. We all watch the same Marvel superhero movies across the globe, and I watch Spanish, Korean and American TV shows on Netflix. The tendency has been for the US to be a cultural pole of attraction. But it has not been a bludgeoning of local cultural at the expense of Hollywood, more we are quickly discovering the universals that drive human enjoyment, and then flavouring with local idiosyncrasies.

“Perceived Immigration”

To many “globalisation” means “immigration”. This appears, at least on the surface, a big part of people’s grievances. However, certain things don’t add up. And the truth is likely more unpleasant, for both sides in the debate.

Firstly, immigration has always been with us. Cheddar man from 12,000 years ago had Finnish characteristics. Neolithic farmers migrated here after the Ice Age 4000 years ago. These were mainly displaced by Belgians around 2000 years ago. These settlers were then supplemented by Romans, Normans, and a huge variety of persecuted groups over the millennia.

Secondly, around 85-90% of Britain is “white” and of “British origin”. This is relatively stable – when recent “increases” in immigration are discussed, we are talking about 1-2% point shifts. This is entirely inline with an increased ease of travel. Hence, “white Britains” are not under threat of “annihilation”, despite the rhetoric. Moreover, nearly every instance of negative sentiment over history is driven by a “race under threat”, which nearly always does not sit with the numbers.

So what is going on?

Let’s break this down. Let’s assume that a perception of increased immigration is real, as this tends to be anecdotally reported by many.

Imagine you live outside the capital of your country (or second/third city or seaboard metropolis). Imagine you are 60. Your experiences of the world are set in late adolescence (~23 according to neuroscience). This means 1981. Now compare 1981 to 2018, what has changed?

First, our cultural horizons have broadened. Local papers, TV news channels and radio stations have all reduced over time to be supplanted by the Internet, and multi-channel Freeview / satellite. The over 60s watch an average of 5 hours of TV a day. A large proportion of these programs are American, which has a different racial composition (60% white, 40% non-white including Hispanic).

Secondly, at least in the UK, we have become much more London-centric. This is true for business, newspapers and TV. London has always been a cosmopolitan city with a much more diverse racial mix. This is now projected onto non-London audience.

Thirdly, the UK media is much better at representation of ethnic minorities, on most channels representation on screen reflects the UK composition as a whole. However, this composition is an average of urban and non-urban environments – cities are normally more ethnically diverse.

This all makes it seem like immigration is rising, even if on the ground there has been less perceived change. Long term trends in globalisation mean that this perception of immigration will not go away, and the move towards a more global culture means that those in more homogeneous areas are going to perceive a greater threat.

Ageing Populations

People are living longer. Our medical success means 80 or 90 is a realistic life expectancy. However, our scientific power has not been accompanied by a similar cultural change in how we evaluate a meaningful life.

As above, we see that neuroscience suggests that life reference points crystallise in late adolescence. This is why you can remember your first albums and romantic liaisons much better than those in your early 30s. It is also why history tends to rhyme, a generation reaches a position of power in its 40s and 50s, but it’s subconscious references points are 20-30 years ago. Hence, it tries to reinvent an ideal imagined future of its youth. In doing this it clashes with the actual experiences of those in their late adolescence at the time, who are imagining their own future without power.

Ageing populations mean with have multiple generations alive at any one time, all with different reference points for life. Those in their 40s and 50s are likely to have active parents, who may shortly need care, as well as their own children. If global connectivity increases a rate of change of material and culturalprogress, then those different reference points become increasingly differentiated.

Those in their old age are also experiencing a rise in loneliness, as ease in movement disintegrates the geographic proximity of family groups: children often live in cities hundreds or thousands of miles away from their parents. Short of (forced) migration, this is likely at odds with a historical reality. Isolation then fuels more extreme opinions, as integration breaks down and it is not possible for the kindness of strangers to nudge you back to the centre.

Healthcare systems also do not have an answer to ageing populations. Look at any chart of spending and you will see it is heavily weighed to those over 65. Anecdotally, I have heard many stories of a range of diseases in the last years of life, heart bypass, stroke, cancer, dementia, pneumonia (typically post 75 or 80). As families may be distant, the burden of care falls on underfunded councils, care workers and hospital staff. These are not equipped to provide meaning to life, or put trials in context.

People are also living longer because they are not dying in childbirth. Read any 18th or 19th century novel and you will see widowers abound, and short paragraphs skipping lightly over the death of several children. This means that not only are we living longer, but each life seems more certain, and may receive more certain emotional investment. It also means we have fewer skills to deal with tragedy.


We are becoming more educated, and so is the world. Many more people are literate, and can engage in current affairs. The majority of the population are enfranchised in a way that didn’t exist 100 years ago.

Education in the UK though is very much a rationalist exercise. It is designed to produce university academics (or Michael Gove). As the population becomes more educated, those that are not suited to this mould fall further behind. This become a problem as those in positions of power with a “perfect” educational background attempt to use advanced rationalist argument to sway sentiment, which just doesn’t work. Many legitimately struggle to understand, and resent the verbal wordplay that they see allows people to say something and nothing at the same time. How often have you seen a politician lead by action?

The education required to be able to have a successful professional career is also changing. I work with inventors in the field of machine learning. Most have PhDs. A career in industry starts at 30. The gulf between the technical expertise needed to produce cutting edge work and the general population is huge. I feel anxiety at not being able to keep up. Transplant that to nearly every modern field, and imagine what it feels like to those with a good, average education, or those that left school at 15 or 16.

Another point is that “talent” is now global. Most successful inventions are invented by 1) teams and 2) inventors from several different countries (often different to the country they are working in). If you are a business, the government or an organisation wanting the best you are, by the laws of statistics, going to have multi-national teams (your net can be cast over the whole world, not just a 10-mile radius). This also adds to perceived “immigration”, those who are successful are likely to be diverse and cosmopolitan. This drives resentment; people may not see those of diverse backgrounds that are like themselves.

It is also a point that those entering the UK from other countries are better educated than the average population. Again this has generally always been the case (the educated middle classes of a country are often able to round up resources to move or escape persecution). For those entering the UK this is a sore point – they often have higher degrees yet are only able to obtain low paying positions. To those already in the UK this is also a sore point – they are likely to see those of different backgrounds progress much faster (as they are promoted towards the natural level of skill based on their education) and see favouritism.


Over human history it is true that there is a general trend towards liberalism. Slavery, torture, hanging, war, and women as chattels have all predominantly faded from view. At the time they faded, debates were fierce. Opinions allegedly shared by a majority have quickly become minority positions; and those on one side of the political spectrum have crossed the chamber many times. Children, women, those of different races and countries of birth, animals, those without a landed estate have all rightly benefited from an equalising in rights.

A great success story of our modern age is LBGT+ rights. That same-sex marriage got onto the statute books of so many countries and states was a piece of good luck. It appears to have wrong-footed conservatives in many places. However, these victories often historically lead to a Newtonian reaction, as those with more conservative opinions feel under attack (which is not helped by the rhetoric of liberals, who often explicitly seek to attack). Progress is more Hegelian than many acknowledge, with a general net progress hiding in the average a zigzagging backwards and forwards.

For example, it appears from several surveys that around 5-10% of the population identify as LBGT+ (with averages being closer to 5% than 10%). This is fairly similar to the racial background figures seen above. Given these percentages, it is not outlandish to think that a population could have many more homophobic people than LBGT+ individuals. While homophobia is wrong, you cannot simply say that homophobia is wrong and it goes away. Opinions change through gradual and frequent exposure to, and positive interaction with, those in a minority; if that exposure and interaction does not take place, the opinions can persist (see Camus’ The Plague for a brilliant viral analogy). On the other hand, telling someone their thoughts and feelings are wrong, more often than not leads to entrenchment rather than tolerance. It is amazing that even after several millennia, classical liberals often forget this. Conservatives perceive a threat to their stable models of the world, to how they make sense of their lives. As such, we need to plan to support liberalisation of societies and to not dismiss those who feel threatened by it.

What are the shorter term changes?

The “Great Recession” of 2008

This is likely the main driver of recent political upheavals, but probably not directly.

The “Great Recession” of 2008 was itself a result of the longer term trends discussed above. German savers investing in collateral debt obligations for US, Spanish and Irish property nearly toppled a human invention built on mutual trust and belief. There is a criticism that the banking institutions were unfairly supported – this is probably true, there was a “heads I win, tails you lose” mentally, but again without bailouts the system may well have collapsed like a house of cards. It could also be seen as a standard down-cycle event (remember “an end to boom and bust”?). In the background we have countries and households supporting increased spending using cheap credit.

There were other factors at play as well. The “Euro crisis” is now often forgotten but was big on everyone’s minds in the years after 2008. This in turn was down to integration difficulties and overheating in an area with monetary but not political union.

The main contributions to popular malaise are twofold: “austerity” and quantitative easing.

“Austerity” came about as countries used to high growth rates driven by globalisation suddenly found a drop off in tax receipts. Borrowing shot up. It seems quaint after nearly a decade of 0% interest rates, but those in the UK Government feared a spike in interest rates as per the 70s/80s. They were saved by the fact that increased global cooperative and correlation meant everyone was in the same boat. As borrowing shot up (a lack of tax income on City bonus pools perchance), governments around the world attempted to apply the brakes to spending. They were pretty clever in doing this. Most cuts wear performed in the background, and tax increases were hidden in things like VAT and a lack of return on saving. While this may have saved political capital in the late 2000s and early 2010s, it began to bite in the mid-late 2010s, as public services experienced funding shortages, and those helped by the state watch their contributions go down (with one exception as discussed later).

Quantitative easing is really a euphemism for printing money. Although much like the stealth of austerity, it was largely hidden behind an technocratic veneer that few could understand. Quantitative easing involved central banks, like the Bank of England, purchasing bonds, typically its own. Now if you or me need to purchase a government bond, we need to pay some money, say £1000 and we get a fixed percentage for a number of years (the coupon or interest rate). After those years are up we get the principal, the £1000 back. To buy bonds for quantitative easing, central banks basically made up that £1000 (*poof* – there is a magic money tree – it’s just a magic money spreadsheet). So central banks effectively printed money to buy assets. They now own trillions of these assets. And generally don’t know what to do with them (the US Federal Reseve may try to sell them this year or next).

Normal money printing is risky it leads to inflation. Germany tried it after the First World War, as did Zimbabwe. Didn’t go well (imagine wheel barrows of notes just to pay for a loaf of bread). The issue is that you have loads of extra money sloshing around (remember those trillions?), so if that hits the real world, people are able to offer increasing amounts for things that are scare and finite in the real world (like bread). However, the idea behind printing money is that you can re-supply money, to get people to start buying things, if they are scared and hiding money under their mattress.

So what was the result of quantitative easing? Unsurprisingly, things that are scare and finite in the real world became more expensive. A big one was housing. Houses in the South of the UK have increased by around 50% in real terms since 2008. Shares are another. Education and social care has also increased by a similar amount. These are also all things that are finite *globally*. London property and private school places are increasingly attractive to a worldwide audience awash with cheap money.

The mechanisms of quantitative easing are ill-understood even by economists. As a buyer of government debt, central banks depressed yields on bonds, lowering interest rates. People with cash and pension funds requiring a safe return were forced into higher yielding assets (which were riskier, hence the paying of a higher return). This meant property to some extent. Many global parties fought over income, rising prices. Risk became artificially depressed.

What didn’t increase? Food, most material goods made abroad, or salaries. Salaries have on average been flat in real terms. This may be due to the constraining effect of austerity on public sector salaries (17% of UK workforce) and the effect of globalisation pushing down the majority of private sector workers. Food and material goods have also benefitted from globalisation and technology advances. They are not “scarce” as they can be relatively freely traded across the world. In Europe, imports from Eastern and Southern Europe offset potential price rises. Also the “Amazon” effect enabled deflation of many goods by cutting out the costs of the middle man. The problem is the middle man had a job and a family to support.

As food and material goods followed salaries, people were not as angry as they could have been. But the slow squeeze from rents, housing, and care costs were slowly felt by many.

Wealthier Pensioners

There is one group who have seen steadily rising incomes in the UK since 2008. These are those over 65. This is partly an effect of the “triple-lock” pension (probably over generous in hindsight), the luck of being alive during times of a booming housing and stock market (since 1991) and the last generation to widely experience benefits such as final salary pensions, which are now phased out for most. Tax changes in the UK, such as the rise in the personal allowance have also taken most pensioners out of the taxation system.

This causes several issues. One is that an older generation has an experience of increasing prosperity, which is at odds with other, younger generations. It is thus more confident, while younger generations are more anxious. Another is that the generation that are most likely to feel anxiety from a pace of social change, are the generation that have the time and the means to be vocal about it.

The iPhone, Social Media & the Decline of Newspapers

What has also changed since 2008? The iPhone and later Android handsets. Over 85% of the UK own a smartphone, a device that didn’t exist a decade ago.

This has effectedly enfranchised a large amount of the population. Many more people now have a voice. It seems that many don’t like this voice. This form of elitism is also regularly found in history, see any debate about extending the vote to non-landed gentry, women and ethnic minorities. The bigger issue is probably that there are so many separate voices, and that in this chaos more extreme positions naturally attract attention (who wants to read about detailed boring technocratic schemes that work?).

This cacophony of voices is correlated with a decline in newspaper readship. Who needs a newspaper when you can get your news and opinions online for free in your pocket? This has two effects. One, newspapers become more desperate to boost readship while needing to cut costs. Journalism suffers. Second, rather than supporting an oligarchy of political positions, so that people modulate their own positions into a known group represented by the paper, political positions are more fragmented and contradictory – the group is forced to shift to the individual’s preference.

Alternate Political Structures

The 21st century has also seen an increasingly successful China and a resurgent Russia.

China has been advancing soft power over the last decade through strategic infrastructure investment around the world. The growth of China suggests to some that democracy or complete political freedom is not required. And it is possibly true that recent Chinese growth could not occur outside of the control and mythology of a one-party state. But China started from a low point of famine, totalitarian rule and repression. It is moving in the right direction but is not at an end-point. Outside of China people often miss the corruption, dodgy construction, and rigging of the system that causes resentment for many. The problem is that China is held up as an example of how we can do away with some of our rights and freedoms and still be a successful society.

Putin is another character that wishes to build a society as imagined by himself in the 1980s. The problem is that society is one built on a KGB framework of information warfare and top-down control. There is a question as to the extent and casual power of recent Russian meddling, but it seems likely there has been meddling. Many of the political trends in the UK appear to mirror those in Russia politics, such as the support of both hard right and hard left positions to create a vacuum in the centre ground that can be exploited and controlled. I wouldn’t be surprised if both Rees-Moog and Corbyn had support and help, maybe obliquely, from Russian state sources. Another Russian KGB strategy is to provide such a chaos of conflicting positions that people are drawn to a position of repression for stability. It is always easier to destroy than create.

This is not to say there are not good people in China and Russia, nor are their aims necessarily nefarious (from the other side there are always valid reasons). The state government in the UK is not synonymous with the views of the population.

Final Thoughts

Political upheaval, and a rise in nationalism, has historically been a way for societies to cope with social change. We have looked above at some of the ways society in the UK has changed over the long and short term. These include globalisation, perceived immigration, aging populations, education and liberalism, as longer term changes, all of them generally positive in the long run, and the “Great Recession”, wealthier pensioners, the rise of social media, the decline of newspapers and a rise in alternate political structures, these having more debatable benefit.

The hope is that we can address the anxiety, fear and anger these changes bring without resorting to fascism, war or increased suffering. I don’t think we can do this through rational argument alone. We need stories that don’t fall back on the easy choice of national identify but that connect emotionally with the reality most people live in. What form these stories take is maybe the next blog post,

Artificial Morality (or How Do We Teach Robots to Love)

One Saturday morning I came upon the website 80000 Hours. The idea of the site is to direct our activity to maximise impact. They have a list of world problems here. One of the most pressing is explained as the artificial intelligence “control problem” : how do we control forces that can out think us? This got me thinking. Here are those thoughts.


The Definition Problem (You Say Semantics…)

As with any abstraction, we are first faced with the problems of definition. You could base a doctorate on this alone.

At its heart, ‘morality’ is about ‘right’ and ‘wrong’. These can be phrased as ‘good’ and ‘bad’ or ‘should’ and ‘should not’.

This is about where the agreement ends.

Let’s start with scope. Does morality apply to an internal world of thought as well as an external world of action? Religions often feature the concept of ‘immoral’ thoughts; however, most would agree that action is the final arbiter. Without getting too metaphysical, I would argue that thoughts (or data routines) are immoral to the extent that they cause physical change in the world in a manner that increases the likelihood of an immoral action (even though that action need not actually occur). For example, ruminating on killing is immoral in the sense that it leads to physical changes in the brain that make a person more likely to kill in future situations.

The main show in morality revolves around the moral groupings: just what is ‘right’ or ‘wrong’? This is where the mud tends to be thrown.

‘Morality’ itself has had a bad rap lately. There are overhangs from periods of dogmatic and repressive religious control. Post modernism, together with advanced knowledge of other cultures, has questioned the certainties that, at least in Europe and North America, supported the predominantly Judeo-Christian moral viewpoint. This has lead to some voices questioning the very basis of morality: if the moral groupings seem arbitrary, do we even need them?

As with other subjects, I think the existential panic that post modernism delivered is constructive for our thinking on morality, but we should use it to build from firmer foundations rather than abandon the building altogether. The body of knowledge from other cultures helps us map the boundaries and commonalities in human morality that can teach us how to construct an artificial machine morality.

Interestingly, morality does appear to be a binary classification. For me concepts, such as an action being half moral or a quarter immoral don’t really make sense. When thinking of morality, it is similarly hard to think of a category that is neither moral nor immoral. There is the concept of amorality – but this indicates the absence of a classification. Hence, morality is a binary classification that can itself be applied in a binary manner.

An Aside on Tribalism

Morality has deep ties to its abstractive cousins: politics and religion. Moral groupings are often used to indicate tribal affiliations in these areas. Indeed, some suggest that the differences in moral groupings have come about to better delineate social groupings. This means that disagreement often becomes heated as definitions are intrinsically linked to a definition of (social) self.

Fear of falling into the wrong side of a social grouping can often constrain public discourse on morality. This is possibly one of the reasons for the limited field size described in the 80000 hours problem profile.

Another, often overlooked point, is that those with the strongest personal views on morality tend to lie on the right of the political spectrum (i.e. be conservative), whereas those writing about morality in culture and academia tend to lie on the left (i.e. be liberal in the US sense). Hence, those writing “objectively” about morality tend to view the subject from a different subjective viewpoint than those who feel most passionately about right and wrong. This sets up a continuing misunderstanding. In my reading I have felt that those on the left tend to underestimate the visceral pull of morality, while those on the right tend to over-emphasise a fixed rules based approach.

Seductive Rules

Programmers and engineers love rules. A simple set of rules appears as a seductive solution to the problem of morality: think the Ten Commandments or Asimov’s Three Laws. However, this does not work in practice. This is clear from nature. Social life is far too complex.

Rules may be better thought of as a high-level surface representation of an underlying complex  probabilistic decision-making process. As such, in many situations the rules and behaviour will overlap. This gives us the causative fallacy that the rules cause the behaviour, whereas in reality similarities in genetics and culture lead human beings to act in ways that can be clustered and labelled as ‘rules’.

This is most apparent at edge cases of behaviour – in certain situations humans act in a reasonable or understandable way that goes against the rules. For example, “Thou shall not kill” unless you are at war, in which case you should. Or “Thou shall not steal”, unless your family is starving and those you are stealing from can afford it. Indeed, it is these messy edge cases that form the foundations of a lot of great literature.

However, we should not see rules of human behaviour as having no use – they are the human-intelligible labels we apply to make sense of the world and to communicate. Like the proverbial iceberg tip, they can also guide us to the underlying mechanisms. They can also provide a reference test set to evaluate an artificial morality: does our morality system organic arrive at well-known human moral rules without explicit programming?

How Humans Do It (Lord of the Flies)

When we evaluate artificial intelligence we need to understand we are doing this relative to human beings. For example, an artificial morality may be possible that goes against commonly-agreed moral groupings in a human based morality. Or we could come up with a corvid morality that overlapped inexactly with a human morality. However, the “control problem” defined in the 80000hours article is primarily concerned with constructing an artificial morality that is beneficial for, and consistent with generally held concepts of, humanity.

As with many philosophical abstracts, human morality likely arises from the interplay of multiple adaptive systems.  I will look at some of the key suspects below.

(Maternal) Love is All You Need

In at least mammals, the filial bond is likely at the heart of many behavioural aspects that are deemed ‘good’ across cultures. The clue is kind of in the name: the extended periods of nursing found in mammals, and the biological mechanisms such as oxytocin to allow this, provide for a level of self-sacrifice and concern that human beings respect and revere. The book Affective Neuroscience gives a good basic grounding in these mechanisms.

This, I think, also solves much of the control problem – parents are more intelligent than their children but (when things are working) do not try to exterminate them as a threat at any opportunity.

Indeed, it is likely not a coincidence that the bureaucratic apparatus that forms the basis for the automation of artificial intelligence first arose in China. This is a country whose Confucian/Daoist morality prizes filial respect, and extends it across non-kin hierarchies.

If our machines cared for us as children we may not control them, but they would act in our best interest.

Moreover, one of the great inventions of the mono-theistic religions of the Middle East, was the extension of filial love (think Father and Son) to other human beings. The concepts of compassion and love that at least Christian scholars developed in the first millennium (AD) had at their basis not the lust of romantic love but the platonic love of parent and child. This in turn was driven by the problem of regulating behaviour in urban societies that were growing increasing distant from any kind of kin relationship.

Social Grouping

The approach discussed above does have its limitations. These are played out all over history. Despite those mono-theistic religions extending the filial bond, they were not able to extend it to all humanity; it hit a brick wall at the limits of social groups.

Although it goes in and out of fashion, it may be that the group selection mechanisms explored by clever people such as Edward O. Wilson, are at play. Are social group boundaries necessary for the survival of those within the group? Is there something inherently flawed, in the form of long-term survival, if the filial bond is extended too far? Or is this limitation only in the constraints of the inherited biology of human beings?

Returning to morality, Jared Diamond notes in The World Until Yesterday that many tribal cultures group human beings into ‘within tribe’ and ‘outside tribe’, wherein the latter are classed as ‘enemies’ that may be ‘morally’ killed. Furthermore, many tribal cultures are plagued by a tit-for-tat cycle of killing, which was deemed the ‘right’ action until the later arrival of a state mechanism where justice was out-sourced from the tribe. We are reminded that “Thou shall not kill” does not apply to all those smitten in the Old Testament.

For machines and morality, this seems an issue. Would an artificial intelligence need to define in and out groups for it to be accepted and trusted by human being? If so how can we escape cataclysmic conflict? Do you program a self driving car to value all life equally, or those of your countries citizens above others? As has been pointed out by many, bias may be implicit in our training data. Does our culture and observed behaviour train artificial intelligence systems to naturally favour one group over another? (Groups being defined by a collection of shared features detected from the data). If so this may be an area where explicit guidance is required.


Marc Hauser in Moral Minds touches on how many visceral feelings of right and wrong may be driven, or built upon, our capacity for disgust.

Disgust as an emotion has clearly defined facial expressions (see the work of Paul Ekman) that are shared across different human groups, indicating a deep shared biological basis in the brain.

Disgust is primarily an emotion of avoidance. It is best understood as a reaction to substances and situations that may be hazardous to our health. For example, disgust is a natural reaction to faeces, tainted foods and water supplies, vomit and decaying flesh. This makes us avoid these items and thus avoid the diseases (whether viral, bacterial or fungal) that accompany them. The feeling itself is based around a sensing and control of digestive organs such as the stomach and colon, the feeling is the pre-cursor to adaptive behaviours to purge the body of possibly disease-ridden consumables.

Hauser discusses research that suggests that the mechanisms of disgust have been extended to more abstract categories of items. When considering these items, people who have learned (or possibly inherited) an association feel an echo of the visceral disgust emotion that guides their decision making. There are also possible links to the natural strength of the disgust emotion in people and their moral sense: those who feel disgust more strongly tend also to be those who have a clearer binary feeling of right and wrong.

This is not to say that this linking of disgust and moral sense is always adaptive (and possibly ‘right’). Disgust is often a driving factor in out-group delineation. It may also underlie aversion to homosexuality among religious conservatives. However, it is often forgotten in moral philosophy, which tends to avoid ‘fluffy’ ‘feelings’ and subjective minefield this opens up.

Any artificial morality needs to bear disgust in mind though. Not only does it suggest one mechanism for implementing a moral sense at a nuts and bolts level, any implementation that ignores it will likely slip into the uncanny valley when it comes to human appraisal.


Another overlooked component of a human moral sense is fear.

Fear is another avoidance emotion that is primarily driven through the amygdala. Indeed, there may be overlaps between fear and disgust, as implemented in the brain. The other side of fear is the kick-starting of the ‘fight’ reflex, the release of epinephrine, norepinephrine and cortisol.

In moral reasoning, fear, like disgust, may be a mechanism to provide quick decision making. Fear responses may be linked to cultural learning (e.g. the internalised response to an angry or fearful parent around dangerous or ‘bad’ behaviours) and may guide the actual decision itself, e.g. pushing someone off a bridge or into a river is ‘bad’ because of the associated fear of falling or drowning, which gives us a feeling of ‘badness’.

Frontal Lobes

The moral reasoning discussed above forms the foundations of our thoughts. The actual thoughts themselves, including their linguistic expression in notes such as this, are also driven and controlled by the higher executive areas of the frontal lobes and prefrontal cortex. These areas are the conductor, who oversees the expression of neural activity over time in the rest of the cortex, including areas associated with sensory and motor processing.

In the famous case of Phineas Gage, violent trauma to the frontal lobes led to a decline in ‘moral’ behaviour and an increase in the ‘immoral’ vices of gambling, drinking and loose women. Hence, they appear to form a necessary part of our equipment for moral reasoning. Indeed, any model of artificial morality would do well to model the action of the prefrontal cortex and its role in inhibiting behaviour that is believed to be morally unsound.

The prefrontal cortex may also have another role: that of storyteller to keep our actions consistent. You see this behaviour often with small children: in order to keep beliefs regarding behaviour consistent in the face of often quite obvious inconsistencies, elaborate (and often quite hilarious) stories are told. It is also found in split brain patients to explain a behaviour caused by a side of the brain that is inaccessible to consciousness. Hence, human beings highly rate, and respond to, explanations of moral behaviour that are narratively consistent, even if they deviate from the more random and chaotic nature of objective reality. This is the critical front-end of our moral apparatus.

Where Does Culture Fit In?

Culture fits in as the guiding force for growth of the mechanisms discussed above. Causation is two-way, the environment drives epigenetic changes and neural growth and as agents we shape our environment. This all happens constantly over time.

Often it is difficult to determine the level at which a behaviour is hard-wired. The environmental human beings now live in around the world has been largely shaped by human beings. Clues for evaluating the depth of mechanisms, and for determining the strength of any association, include: universal expression across cultures, appearance in close genetic relatives such as apes and mammals, independent evolution in more distant cousins (e.g. tool use and social behaviour in birds), and consistency of behaviour over recent recorded time (10k years).

My own inclination is that culture guides expression, but it is difficult if not impossible to overwrite inherited behaviour. This is both good and bad. For example, evidence points to slavery and genocide as being cultural, they come and go throughout history. However, it is very difficult to train yourself not to gag when faced with the smell of fresh vomit or a decaying corpse.

A Note on Imperfection

Abuse. Murder. Violence. Post-natal depression. Crimes of passion. War. Things can and do go wrong. Turn on the news, it’s there for all to see.

Humans have a certain acceptance that humans are imperfect. Again a lot of great art revolves around this idea. People make mistakes. However, I’d argue that a machine that made mistakes wouldn’t last long.

A machine that reasons morally would necessarily not be perfect. To deal with the complexity of reality machines would need to reason probabilistically. This then means we have to abandon certainty, in particular the certainty of prediction. Classification rates in many machine learning tasks plateau at an 80-90% success rate, with progress then being measured for years in fractions of a percent. Would we be happy with a machine that only seems to be right 80-90% of the time?

Saying this I do note a tendency towards expecting perfection in society in reason years. When something goes wrong someone is to blame. Politicians need to step down; CEOs need to resign. There are lawsuits for negligence. This I feel is the flipside of technological certainty. We can predict events on a quantum scale and have supercomputers in our pockets; surely we can control the forces of nature? Maybe the development of imperfect yet powerful machines will allow us to regain some of our humanity.