Machines of no intrinsic grace

Dario Amodei’s influential paper, Machines of Loving Grace, illustrates AI’s potential benefits across areas as diverse as biology, neuroscience, the economy, governance, and work. On governance, Dario rightly argues there is “no strong reason to believe AI will preferentially or structurally advance democracy and peace [...]. It’s therefore up to us as individual actors to tilt things in the right direction."

In this essay, I argue that AI’s contribution to democracy will depend on whether AI’s benefits are widely distributed. I tell two distinct hypothetical tales: one where AI empowers autocratic tendencies then another where AI strengthens democratic forms of governance. I provide illustrative examples of ways AI might be used in both scenarios and show how that process could unfold in phases, starting with initial AI applications before progressing to more structural impacts. I end with my central argument,that the path societies land on depends largely on how many get to reap the benefits of AI.

Autocracy as the end

Let’s first embrace a pessimistic worldview where the ascent of AI accelerates democratic backsliding and empowers autocracies, returning world history to the form of government it is most familiar with. I’m making here a subjective prediction for how autocracies could hypothetically develop or be strengthened as a result of AI. The prediction is based on my general interest in technology as well as geopolitics and history, but I am not a PhD. I structure the prediction in three stages for clarity.

AI blurs the line between real and unreal

In 2024, the world got a preview of the ways AI might lower information integrity beyond the damage already caused by social media. During the New Hampshire Democratic primaries, some Democratic voters received a phone call from U.S. President Joe Biden urging them not to vote. In Slovakia, during the parliamentary elections of September 2023, just hours before the vote, a fake AI-generated interview circulated widely. In it, the leader of the centrist and pro-European party Progressive Slovakia, Michal Simecka, can be heard conversing with a well-known investigative journalist about how the vote will be manipulated to favor his candidacy. Even though the video was removed by social media platforms a few hours after being posted, it had already been shared several thousand times. While the reach of both interference attempts was limited, media coverage was significant enough to raise the “liar's dividend” concern: the possibility for politicians to one day get away with any“ scandal”, by claiming it was or was not true.

Another observation about those two uses of AI in an electoral context is future interference attempts will likely be more effective, as AI models become more persuasive and campaigners learn to personalize their approach. Imagine the same phone call from U.S. President Biden in 2024 with a more human-sounding voice, the ability for the AI to react to the targeted voter’s interjections, and to personalize its responses based on what it knows about the voter. Most of these AI capabilities are already available but they were built in 2024, too close to the US election to impact it (as far as we know). We should assume they could be used in future elections.

AI-powered monitoring enables totalitarian control

A sign that a democratic government is further ‘backsliding’ could be that it starts to use AI’s data gathering and analysis powers for efficient monitoring and surveillance of civilians. As Henry Farrel puts it, “when every transaction that a citizen engages in is recorded by tiny automatons riding on the devices they carry in their hip pockets, when cameras on every corner collect data on who is going where, who is talking to whom, and uses facial recognition technology to distinguish ethnicity and identify enemies of the state, a new and far more powerful form of authoritarianism will emerge”.

In this phase, AI could be used to not only anticipate but also silence dissent. Suppression of protests could become automated, using systems that are more effective than humans, making public revolt less threatening. Attempts to seek reparation in court could be thwarted by AI legal agents that flood the courts with appeals and counterappeals, paralyzing the judicial system.

AI-driven automation eliminates labor bargaining power

In the end, the most pernicious and long term impact of AI might be a removal of incentives for elites to share economic or political power. To understand how this might play out in practice, let’s go back to the time in History when modern democratic systems emerged. Democratic renaissance in the modern era came with the industrial revolution, when factories powered by the invention of the steam engine and railways amalgamated previously separate, small communities of farmers and townspeople into the densely populated neighborhoods of large cities, leading to the birth of a large class of salaried employees with collective bargaining power. As Research Fellow at the University of Oxford Ben Garfinkel, argues, AI automation could lead to fewer jobs and higher demand for work, thereby diminishing salaried employees’ bargaining power. If employees can no longer afford rent in densely populated cities, this could in turn lead to a more distributed population that is less likely to strike, organize, and protest. If and when protests and strikes occur, the incentive for elites to redistribute power or income would be low. Autocratic systems could become entrenched, with no real incentive to go back to democratic forms of governance.

Democracy as the end

It doesn’t have to end this way.

Let’s explore a more optimistic worldview where the rise of AI encourages the return or strengthening of democratic forms of government. To that end, we need to go back to a time in History when a technological breakthrough contributed to an era of liberalisation and democratization: the invention of the printing press in the 16th century. By making information easier to produce and distribute at scale, the printing press helped spread the ideas of ‘enlightened’ thinkers like Rousseau, Diderot and Adam Smith and beyond. While critics like Yuval Noah Harari argue the disruptive force of the printing press also led to violence - Europe experienced almost a century of religious persecutions in the 17th century - the ethos of Silicon Valley is still very much imbued with the belief that more knowledge leads to more empowerment and freedom, including in the political sphere.

Let’s look at examples of how AI could power a democratizing dynamic.

AI streamlines bureaucracies and improves access to government services.

Democracies around the world could endure not by resisting AI but by integrating it into governance to strengthen institutional resilience. In Machines of Loving Grace, Dario Amodei argues “there is a clear opportunity for AI to be used to help provision government services—such as health benefits or social services—that are in principle available to everyone but in practice often severely lacking, and worse in some places than others.” Let’s look at examples of how AI could be used to streamline bureaucracies and improve access to government services.

• AI could give citizens information about all the benefits they’re entitled to in simple and human language—and also help them comply with often confusing government rules.
• AI agents could alleviate the burden on agencies like the IRS by managing inquiries, with instant translation services to extend their reach to low literacy groups, minority language groups, and people in crisis.
• AI could help lawmakers simplify laws, contrast & compare legislation, generate legal documents, transcribe proceedings and distill complex legislative documents and policies into understandable summaries for the general public.
• AI could also help Judges make better, fairer, and faster decisions by helping executive branch leaders quickly analyze hypothetical scenarios.
• As a whole, AI could help the different branches of government meet people’s raised expectations, reduce the gap between public and private sector capacity, restore trust in government and even boost voter turnout.

An important success criteria would be the extent to which, in the process of developing AI applications, governments can balance the need for efficiency with a commitment to fairness and inclusion, so they can honor their mission to serve the general public.

AI enhances civic engagement

One of the most exciting opportunities for AI to play a democratizing role will be in the facilitation of enhanced civic participation, provided applications unlock political impact and are not solely performative.

I break out examples in the categories of education, representation and deliberation.

• Education: government agencies and civil society organizations could use AI to create compelling, non partisan voter information campaigns and adaptive learning experiences tailored to individual users, covering topics like democratic principles, civic responsibilities, and critical thinking skills. AI could help translate materials instantly into minority languages, and even break conspiratorial rabbit holes.
• Representation: elected representatives could deploy agents to understand their constituents better or to conduct sentiment analysis. School administrators could deploy chatbots to collect feedback and suggestions from their students. AI moderated forums could increase participation from the “silent majority”, helping paint a more comprehensive picture of participants’ ideas and concerns.
• Deliberation: AI could be used to drive consensus, find common ground, seek compromise or resolve conflict among citizens. Some early ideas in this direction have been undertaken by the computational democracy project, which collaborates with Anthropic. AI could help large groups to engage and deliberate about how best to address common problems, help citizen assemblies summarize proposals, help identify common understandings among large and diverse publics, make deliberation democratically viable at scale, facilitate deliberative democracy through AI-generated discussions, transform civic discussions into structured narratives for better comprehension, and even support participatory urban planning.

If such applications are widely adopted, they could contribute to a more informed, engaged and represented citizenry.

AI monitors for signs of democratic backsliding

Just like AI could be used by autocrats to surveil citizens, citizens could use AI to keep their government under close watch.

Imagine an AI agent that has been programmed by citizens or opposition parties to abide by values of individual freedom and to carry out actions that support the preservation of democracy. The agent could infiltrate an autocratic government’s IT systems to sow chaos, it could produce messaging and content that looks like it’s coming from the government, to ridicule it or to call its bluff and it could create spam to slow down objectionable government operations. Every engaged citizen could also carry a version of it in their pocket, for support and advice at all times.

AI’s data analysis powers could also be used for monitoring. It could spot patterns in governance, media freedoms, and civil liberties in order to anticipate potential democratic backsliding. It could also be used to monitor human rights abuses or detect irregularities that may indicate corruption or misuse of funds.

Common patterns

As I look back at all the ways that AI could be used for autocratic or democratic ends, I am struck by the following similarities:

• AI is dual-use. Like all general purpose technology, it can empower defenders as much as attackers of democracy.
• There is no fatality. Whether the same AI tool is applied for the purpose of surveilling citizens or summing up citizen concerns, is a function of developers’ agency and any opinionated guardrails they might add to their system’s design.
• The most immediate uses of AI for governance are likely to be for productivity, whether bad actors use AI to try to improve their targeting, or government services use AI to be more lean
• Once AI habits become entrenched in a democratic or autocratic systems, the switch to another form of governance could become less likely. AI powered surveillance in an autocracy could make a switch to democracy harder just like AI powered activism in a democracy could make a switch to autocracy harder. This could mean that we witness over time a new form of polarization where established democracies become more democratic over time and established autocracies become worse.
• AI is a tool that, for as long as it is“aligned”, will follow human intent. Put it in a society that’s rapidly democratizing and its effect could be to accelerate that trend. Put it in a society that’s going in the opposite direction, and it could amplify that process instead. In other words, one of the largest pre-determining factors to the direction AI takes is the health of pre-existing institutions.

The impact of AI on governance will not hinge on how AI is used, but on how many reap its benefits

It is impossible to predict whether, 10 years from now, we’ll look back at the impact of AI over the past decade as having been fundamentally positive or negative for democracy. Illustrating the ways things could tiltdoes not answer the question of likelihood of either scenario, and it is also very possible we see non homogenous effects across the world, depending on the situation each country finds itself in at the beginning of the AI era.

My view is that whether we find ourselves on an autocratic or a democratic path will depend on the extent to which AI’s main effect is to distribute or to further concentrate economic and political power.

To understand why, let’s look at the times in modern history when patterns of wealth distribution significantly influenced the nature of governance.

• On the whole, periods characterized by more equitable wealth distribution have often coincided with the strengthening of democratic institutions. One example is the rise of the working class in the 1850s. This period saw labor movements advocating for better wages, reasonable working hours, and improved conditions. These movements led to significant democratic reforms, including the expansion of voting rights and the establishment of labor rights, thereby strengthening democratic governance.
• Another example is the rise of the middle class in the 1950s in many Western countries. The post WWII era, often referred to as the "Golden Age of Capitalism," was characterized by policies that promoted economic equality, such as progressive taxation and social welfare programs. The widespread prosperity and the presence of a robust middle class contributed to political stability and the deepening of democratic institutions.

On the other hand, eras marked by pronounced wealth concentration have frequently seen the rise of populist and autocratic regimes.

• The Great Depression of 1929 led to massive economic upheaval, resulting in significant wealth concentration as businesses failed and unemployment soared. In Europe, economic distress contributed to the rise of totalitarian regimes in Germany and Italy, where leaders capitalized on public discontent to dismantle democratic institutions.
• The 2008 financial crisis resulted in significant economic downturns globally, leading to increased wealth inequality. The crisis's economic hardships contributed to the rise of populist sentiments in various nations, leading to the election of leaders with autocratic tendencies.

We can therefore argue AI’s effect on wealth distribution could have very clear political outcomes.

• If AI’s benefits are distributed across all levels of government, local to state to federal, and across the world, these different “units” of political power could be better equipped to resist the influence of a unique, central autocrat.
• If AI’s benefits are distributed across citizens, including access to intelligence and consensus-building tools, we could see the beginning of a herd immunity against micro-targeted propaganda, and the hyperpolarization autocrats feed off of.

Realistically, this redistribution will be hard. The economic benefits of globalization, and later of the Internet Age, have not been shared as much as one may have hoped.

• In 1989, the top 1% of U.S. households held approximately 23.3% of the nation's wealth but by 2022, this share had risen to nearly 35%.
• In 1965, the average CEO-to-worker pay ratio was 20:1 but by 2022, this ratio had escalated to 400:1, indicating that CEOs now earn 400 times more than the average worker.
• Between 1979 and 2016, after-tax income for households in the top 1% grew by 226%, while income for the bottom 20% grew by only 85%.

Such unprecedented levels of wealth concentration have no doubt played a role in recent democratic backsliding. With AI, we get another chance to try to distribute the cards and there’s a role for everyone to play:

• AI developers need to think one step beyond growth metrics, and consider who ultimately benefits from their applications and how.
• Individuals need to think of AI tools as tools for political and civic empowerment, in addition to economic empowerment.
• Civil society including press and non governmental organizations need to think one step ahead of governments, and leverage AI for reporting, monitoring, advocacy and mobilization.
• Governments need to create the conditions, including the regulatory frameworks, for broad distribution of AI within some core safety guardrails. Democratic governments further need to embrace AI for trust rebuilding efforts, starting with a streamlining of government services.

Over the past 3o years, democratic backsliding has removed oxygen from our world, as if each democracy that fell were a tree cut to the root or worse, rooted out. Slowing down or reversing this process will not happen overnight, but the seeds for it need to be planted today.

*Notes

1. Nothing in this essay attempts to reflect, represent, or challenge the views of my current or former employers. This essay only reflects my own views.
2. By “distributing AI’s benefits”, I mean the economic value AI creates in the form of productivity or increased wages. I do not mean simply “increasing adoption of AI tools”
3. I am not looking in this essay at the set of incentives each player in the equation - civil society, governments, citizens/users - has for distributing or concentrating AI’s benefits. This could be the subject of another essay.
4. I do not believe the deciding factor for whether AI benefits democracy is whether AI will undermine or promote truth. By focusing on the eroding lines between ‘truths’ and ‘falsehoods’, we refuse to accept the reality that we have entered an era where no amount of fact-checking can correct “lies" when lies have less to do with the truth and more to do with power. In other words, the question of whether AI in the end benefits democracy or autocracy is not about whether it leans towards “truth”. It is about who it gives more power to.*