Lock-in is a central theme to this organisation. Therefore, before conducting any research on lock-in, it is important to create a strong working definition of lock-in for use in all subsequent products. The stronger the definition of lock-in, the stronger the foundation on which the subsequent work will rest, and the more specific and explicit we can be when talking about variations of lock-in.
MacAskill talks about lock-in in a chapter of What We Owe the Future. He uses the Hundred Schools of Thought period in ancient China, the concept of writing, and the potential for artificial general intelligence to define and make an argument for the potential for value lock-in. He defines lock-in as a situation where a single ideology or set of ideologies, or value system takes control of an area of the world, or the whole world, and persists for an extremely long time.
In his chapter on future risks, Ord categorises potential existential catastrophes for humanity into extinction, and failed continuation. He then categorises failed continuation into the unrecoverable collapse of civilisation, and unrecoverable dystopia.
He then divides the unrecoverable dystopias into three categories:
In an interview with Future Matters, Lukas Finnveden, main author on AGI and Lock-In, and research analyst at Open Philanthropy, defines lock-in in terms of predictable stability. He states, “Some property of the world is locked in if it's very probable that the property of the world will hold true for a very large amount of time“.
Jess Riedel, co-author of AGI and lock-in, physicist at NTT Research, uses the term singleton introduced by Nick Bostrom in “What is a Singleton?“ to create a definition of lock-in. If there is an AGI singleton (world order in which there is a single decision-making agency at the highest level) with stable values (its values do not change over time), then those values are locked-in. This can be further defined as global lock-in if it applies to all earth-originating life.
Carl Shulman, co-author on AGI and lock-in, has also done extensive thinking on lock-in, but does not provide a concise definition in his public discussions.
An example of technology lock-in is the QWERTY keyboard. This concept illustrates how technologies can become locked in by markets and evolutionary forces, leading to stability that is not necessarily optimal.
Moloch is a symbol of systems that demand constant sacrifice, causing the depletion of human potential. It serves as an analogy for value lock-in, where society can be locked into values which require continuous sacrifice, e.g., resource exploitation requiring constant environmental degradation. Scott Alexander lists 10 real-world examples of multipolar traps in “Meditations on Moloch“ to illustrate this analogy.
A static society is characterised by minimal change or evolution over long periods. In “The Beginning of Infinity“, David Deutsche describes societies as static (values held in place with little variation for long periods) and dynamic (inversely, values updating in the direction of what we call progress over short periods with no defined endpoint).
The Malthusian Trap refers to self-reinforcing dynamics that can be likened to lock-in dynamics. Resource depletion limits growth, and constrains progress and improvement without paradigm-shifting innovation.
Concepts from economics and game theory such as vendor lock-in, tragedy of the commons, path dependence, equilibrium lock-in, repeated games, and deadlocks also provide models for thinking about how situations can get locked in.
For our purposes, we define lock-in as a situation where a feature of the world is stable for a long time. Our definition of lock-in follows from, and is most aligned with, Lukas Finnveden's definition because that definition targets the long-term and neutral nature of what we consider to be a lock-in, and we believe this is the best explanation for what a lock-in is.
Following on from this definition, we can observe that many things can be considered lock-ins. For example, Earth's temperature is a feature of the world that has been stable for a long time. But we don't mind that feature being locked-in. Therefore, along with our definition, we need to define our scope of interest.
The things we care most about when wanting to minimise lock-in risks (from our point of view as humans) are not all the features of the world, but the features of human culture, namely values and ethics, power and decision-making structures, cultural norms and ideologies, and scientific and technological progress. From the perspective of minimising risk, we are therefore most interested in potential lock-in scenarios that would be harmful, oppressive, ultimate (long-term or final), parochial, or large.
So now the definition looks something like bad situations for our culture that last a long time. But what about Ord's focus on desirability, and Jess Riedel's focus on AGI? Our definition now attempts to unify these attributes by categorising lock-ins. Let's start by distinguishing between positive, negative, and neutral lock-ins.
Positive lock-ins are tricky, because as MacAskill points out, today's positive lock-in might be considered parochial and suboptimal by future generations. Just as locking in the values of slavery would be seen as a terrible idea today, so locking in some values of today might be seen as a terrible idea in the future. This is ultimately a paradox, because in a society that makes constant progress towards an improved version of itself, there may never be a point at which we can be comfortable locking in anything.
However, there is a small space of potential solutions where we might be able to converge on something close to positive lock-ins. This is the region where we lock-in features of human culture that we believe contribute to the minimisation of lock-in risks.
We define these as lock-ins that we are not particularly interested in. As mentioned, the openness of our definition allows for many features of the world to be considered lock-ins. For example, the temperature of Earth, or more specific to human culture, the concept of work. These are features of the work which tend to remain stable, but that we are not trying to make more or less stable.
These are the lock-ins we are most interested in. The formation of these kinds of lock-ins would have negative implications for humanity. As mentioned, we care most about lock-ins that are:
So we have established a categorisation of lock-in in terms of the desirability of outcomes for individuals. Now let's factor in AI. As most key thinkers identify, AI, and especially AGI, plays an important role in hypothetical future lock-ins. There are many reasons for this, such as improved surveillance, error correction, human longevity and immortality, and power-seeking behaviour and instrumental convergence. It is important to address this fact by further categorising lock-ins by their relationship to AI systems. We created three such categorisations:
This section outlines the existing literature and thinking on lock-in, highlighting the contributions made by key thinkers in the area, and adjacent concepts. A definition of lock-in is then synthesised in light of the existing definitions and goal of the proposed organisation, and categories are defined to make different forms of lock-in explicit in terms of scope, desirability, and AI. This section aims to be the starting point for further work on lock-in risks by creating a foundational definition of lock-in that such work can use.