Capital Ownership Will Not Prevent Human Disempowerment

When discussing the future of AI, I semi-often hear an argument along the lines that in a slow takeoff world, despite AIs automating increasingly more of the economy, humanity will remain in the driving seat because of its ownership of capital. This world posits one where humanity effectively becomes a rentier class living well off the vast economic productivity of the AI economy; where, despite contributing little to no value, humanity can extract most/all of the surplus value created due to its ownership of capital alone.

This is a possibility, and indeed is perhaps closest to what a ‘positive singularity’ looks like from a purely human perspective. However, I don’t believe that this will happen by default in a competitive AI economy, even if humanity goes into the singularity owning all of the capital and there is no dramatic upheaval and everything evolves peacefully with capitalism and rule of law is preserved. One historical intuition pump I have around this is what happened to the feudal aristocracy when the Industrial Revolution occurred. The defining feature of almost all societies prior to the Industrial Revolution has been dominance by a small landowning class — an aristocracy — who own almost all the ‘capital’ in the society, who also control its politics and exist primarily as rentiers. What such a scenario proposes is that humanity effectively becomes an aristocracy upon a highly productive AI population.

However, historically, despite a landowning aristocracy being the dominant form of human society throughout history until then, over merely 100-200 years of the Industrial Revolution, aristocracy faded from being the dominant force in society to being at best a minor player and at worst completely irrelevant. Moreover, today the degree of aristocratic control is highly correlated with being a poor country that did not fully industrialise and which ended up being outcompeted by those countries that did. While in some countries this de-aristocratization process was violent, the same trend occurred peacefully in many others. Theoretically, if capital stock is all you need, the aristocracy entered the Industrial Revolution in the perfect position to maintain and consolidate their power, given that they controlled both the major capital stock of the economy at this time, and that, like the current AI revolution, the Industrial Revolution intrinsically required huge capital investments in factories, machines, infrastructure, railways, canals etc. The aristocracy additionally controlled the political system. Nevertheless, they entirely failed to maintain their relative power 1.

It is very instructive to consider this historical parallel and try to consider why the feudal aristocracy failed to maintain control during the industrial revolution and whether indeed they could have at all. There are a number of deep reasons why their control slipped and many of these factors are also present today, and are very important to understand well if we try to exert control over the shape of the singularity. My analysis of the factors that make it extremely challenging for ownership of capital to ensure long term control under a new economic paradigm are as follows:

1.) Changes in the form of capital : The industrial revolution made fundamental changes to the economy and radically changed what was meant by ‘capital’. Prior to the industrial revolution, capital had always been in the form of land which was primarily used for farming. The major human capital was the tenant farmers who worked the land. Productivity of a patch of land varied, but was generally known and only varied across a small range. While there was always trade which utilized capital in the form of inventories of trade-goods, and ships, this was a relatively minor part of the economy and was much more risky than land. The industrial revolution created new forms of capital mostly de-novo in the form of factories, complex supply chains, complex infrastructure etc. To build this required much existing capital investment and reallocation of resources away from existing economic activity. However, the returns, while often very variable, were much higher than buying land and vastly more scalable. This allowed industrialists, who grew their capital from a much lower base to rapidly eclipse the existing landowning class in effective wealth. The parallels to the singularity are clear. While today, capital exists in many forms, including ownership of land, of intellectual property, ownership in corporations, etc, it is likely that the singularity will bring about novel forms of capital which have, at least initially, much greater returns and scalability than possible in today’s economy. This could include ownership of minds, of AI systems, of compute power, and of many other things very hard to predict today. Having majority ownership of the capital of today does not guarantee succeessful...