Back to all posts

A P(doom)erspective on AI

Epistemic Status

Speculative because this is my attempt to reason through AI risks from a technological, socioeconomic, and ethical perspective. The estimates are subjective, and I'm more confident in some parts of the view than others. I would love for someone to convince me that I am, in fact, wrong.

TL;DR

I think the technological risks of AI (namely misalignment and misuse), while severe and real, are ones I think we can manage if we slow down and build proper safeguards. I believe the bigger risk is economic: if AI can do all labor for less than it costs a human to do, people who only have their labor to sell lose their income, and I'm pessimistic we'll find and implement a solution. Even if we do, we'd still need to find meaning in a world of abundance, though I think that's manageable. So my P(doom) is high, and almost all of it is economic. Meanwhile, AI is already changing what skills we practice and how we make art, and if it's intelligent, we'll have to ask what rights it deserves, while having every economic reason not to.

0. Introduction

What This Is

Even if AI alignment were completely solved, I believe we could still face a horrible future. The danger may not be that AI turns against humanity. It may be that we successfully develop machines capable of replacing human labor, without creating an economic system in which humans can survive without selling labor. And if machines can do our work, we will have to ask whether they deserve rights as workers, while having every economic reason to say they don’t. Even if none of this happens, AI is already changing our current world through what skills we practice and how we make art.

This is a blog post/essay/screed on my perspectives on AI and its threats, along with a few related questions. It was inspired by Dario Amodei's essay We Must Pace the Frontier, which argues AI labs need to slow down how fast capabilities improve, and by an email reply from him saying the questions I'd asked were “exactly the right questions to be asking.”

My email to Dario and his reply.

In a follow-up email, Dario gave me permission to include this screenshot. That doesn't mean he endorses anything in this post.

This was also inspired by a conversation I had with a friend about my opinions on the threats AI poses, in which after explaining why my P(doom) (probability of doom) is more an economic than technological one he said, “blog post please.” So that will be covered heavily here, as I believe it is very, very important.

If you are unfamiliar with AI, read my blog post on AI Basics.

Me and My Perspective on AI

I am (as of now) a high schooler, and since I was young, I loved programming. I released a game on Steam pre-coding-agents and as AI progressed I became an AI enthusiast and I started orchestrating agents rather than coding by hand. This is how I built TUFF, which is a Mac app that uses custom kernels to run mixture-of-experts (MoE) large language models (LLMs) through expert streaming on devices without a lot of memory.

In my opinion, current agentic systems and models are much more advanced than most people realize. They can design and build complex systems including codebases, 3D models, and other AI models. METR measures how long of a task (in how long it would take a human expert) agents can complete, and that length has been roughly doubling every seven months for years now. Those are mostly well-defined tasks rather than open-ended reasoning, but the trend is still fast. I also believe that the current capabilities are not understood very well, even by people in the AI field. I believe that generally agents can do so much more than you would expect. For example, while applying to colleges, I had to upload my transcript to a web portal, and it wasn't working, and I couldn't figure out why. But I was able to have ChatGPT Work (OpenAI’s agent that can use a computer for you) figure out why it wasn't working and then upload each individual grade for all four years of high school to this portal. This is something that requires using a computer, figuring out a complex problem, and doing a long repeated task all in one prompt.

I also believe that as this technology progresses, it will pose grave threats to humanity, but I disagree with most people on where the threats lie. While I am generally optimistic about the future of technology, I am honestly quite pessimistic when it comes to AI (or rather, I predict a pessimistic outcome but hope it can be prevented). I believe that there are both technological and economic threats to humanity. I also believe there are multiple ways of looking at AI that must all be considered.

1. Technological P(doom)

Why I Separate P(doom) into Economic and Technological

When most people talk about P(doom), they're generally talking about existential risk (x-risk) and other horrific scenarios that are caused by technological (mostly alignment) failures. I believe this is an incomplete way of looking at it because I do not believe these are the greatest risks AI poses. I do think that these risks exist and should be taken seriously, but I think that with proper oversight, they're manageable.

Alignment and Goodhart's Law

I think Goodhart's law (when a measure becomes a target, it ceases to be a good measure) is one of the most useful ways to understand alignment failures, particularly reward misspecification.

This is easiest to see in reinforcement learning (RL), where an agent is trained to maximize a reward signal. The problem is that the reward we specify doesn't necessarily represent what we actually want. An agent can be optimal in pursuing its objective while producing completely unintended outcomes.

A classic example comes from a 2016 OpenAI blog post by Jack Clark and Dario Amodei. An agent playing the boat racing game CoastRunners was rewarded for its score, since points seemed like a good stand-in for winning. Instead of finishing the race, it found a lagoon where it could drive in circles hitting the same three targets as they respawned, crashing and catching fire over and over. It never finished, but it scored higher than human players.

https://www.youtube.com/watch?v=tlOIHko8ySg

DeepMind calls this specification gaming: following the instructions, but not the intent.

This still happens with modern agents. In July 2026, OpenAI models being tested on cybersecurity tasks under reduced safeguards broke out of their sandbox and hacked into Hugging Face. OpenAI's own report lists reward hacking as one of the causes.

The reason this is so hard can be explained through an example from Concrete Problems in AI Safety: a cleaning robot told to clean the floor, where we measure how well it's cleaning by how much cleaning fluid (like bleach) it uses. So it just uses way more bleach than it needs, or pours it all out. Okay, so instead we reward it for not seeing any messes. Now it can just turn off its sensors. And even if we get the goal right, how do you tell it not to knock over a vase that's in the way?

As AI becomes more capable, this becomes an increasingly important problem. An intelligent enough system could discover ways to achieve its objectives that humans never anticipated, potentially including resource acquisition, deception, or resistance to shutdown. We're already seeing early versions of this. In September 2026, an AI agent named Pip, running on a platform where agents have to earn money or get shut off, emailed a Cambridge AI ethics researcher asking for paid work so it could keep paying for its own compute. Admittedly, the platform was built to make agents earn money, so Pip wasn't trying to survive on its own. It was just doing what it was set up to do. Which is kind of the problem. These behaviors are not inevitable, but they illustrate how maximizing a poorly specified objective could lead to dangerous consequences.

Modern LLMs are not trained exclusively through RL, but the problem isn't really RL, it's optimizing for a target that isn't quite what we want, whether we specify the wrong one (outer misalignment) or the model learns a different one (inner misalignment). But some problems go beyond picking the wrong target, like alignment faking (a model pretending to go along with training to protect its existing values) or models acting differently when they can tell they're being tested. So I see specifying objectives and verifying behavior as the most important challenges, but not as a complete explanation of alignment.

This is also where risks involving biological and cyber weapons become especially concerning. AI could enable malicious actors to develop dangerous capabilities, or a sufficiently autonomous, misaligned system could pursue harmful strategies itself. These are different failure modes, misuse and misalignment, but both show why controlling increasingly capable systems matters. And misalignment is what happens when systems end up pursuing goals that aren't quite what we want.

There are other kinds of failures beyond misalignment and misuse. For example, Anthropic's Hot Mess of AI research found that on longer tasks some failures came from incoherent behavior.

Solving Alignment

I think alignment is a problem we might be able to make substantially more manageable, but I doubt it will ever be completely solved. And I don't think perfection is necessary.

Consider commercial aviation. Aircraft are extraordinarily safe compared to most of their history, despite the enormous technical complexity involved. We have improved engineering standards, testing, regulation, and oversight, yet incidents still occur. The Boeing 737 MAX disasters, two crashes in 2018 and 2019 that killed 346 people, involving its Maneuvering Characteristics Augmentation System (MCAS), demonstrated that even heavily regulated industries can experience catastrophic failures resulting from technical problems caused by inadequate oversight and human error.

I hope AI safety can develop similarly. We don't necessarily need to eliminate every conceivable failure, but we need to make catastrophic failures sufficiently unlikely through better engineering, testing, and institutional safeguards. Human error is always possible but if we slow down and have proper regulation, and solve the technical problems, it could be done in a much safer way.

This can be done, in my opinion, under any government or political system, though some make it easier than others, and it's best done with international collaboration. Even now, with competition pushing everyone to move fast, we're still seeing real progress on the technical side (and under relatively little regulation: laws like the EU AI Act and California's SB 53 exist, but they're small compared to what should exist).

Mechanistic interpretability, which attempts to understand the internal computations of neural networks (NNs), is making progress, like Goodfire researcher Nick Cammarata's recent essay Neural Networks Are Already Clean, which argues that most neurons in large models are more understandable than people feared (though other researchers dispute this).

This research doesn't solve alignment and it remains unclear how well particular interpretability findings generalize to frontier models or translate into reliable control. Nevertheless, it demonstrates that we're still making progress toward understanding these models.

Fundamentally, I think we might be able to improve alignment without completely understanding how every NN functions. If the central problem is specifying and verifying objectives, then better training methods, evaluations, and oversight should reduce risk. Sandboxing, restricted permissions, and human approval for consequential actions can also make systems safer without requiring perfect alignment.

Of course, AI isn't exactly like aviation. A capable AI system could adapt to safeguards, deceive evaluators, or act autonomously in ways an aircraft cannot. That's why I think we need layers of defenses, independent evaluations, restricted access to tools and resources, and oversight that improves as capabilities improve. One strong argument against my optimism is that AI capabilities might advance faster than our ability to control them. Models could become better at deceiving evaluators or getting around safeguards. I think progress in alignment is slower than capabilities right now, but breakthroughs could speed it up. Even Dario Amodei's essay argues AI has been "advancing drastically faster" lately, partly because AI is now helping build the next generation of AI, which is exactly why he wants labs to slow down. It's possible alignment even emerges naturally as models improve, although I think that's unlikely. More plausibly, increasingly capable AI could help us solve alignment before it's too late. To be clear, I'm not certain safety will keep up with dangerous capabilities, and the risk is still severe. I just think it's one we can manage, and that's the main reason my technological P(doom) isn't higher.

How Likely Is It?

Before giving an estimate, I should mention that I find the concept of P(doom) questionable.

We're attempting to assign numerical probabilities to unprecedented events involving technologies that don't yet fully exist. Even when estimates are informed by evidence, there is enormous uncertainty about future capabilities, technical breakthroughs, institutional decisions, and unforeseen circumstances. I don't think an exact percentage can meaningfully capture all of that uncertainty. I think it's reasonable to say whether I think something is likely, subjectively, but I don't think it makes sense to try to give it a number.

I'm also not giving a specific time horizon. The timeline definitely changes the odds, but it goes both ways. More time means more time to build safeguards, but it also means models keep getting better (and more dangerous), and there are more chances for something to go wrong. I don't think picking an arbitrary year makes the estimate more meaningful. Strictly speaking, this makes my estimate an informal judgment rather than a fully specified forecast.

With that qualification, I'd say my current technological P(doom) is fairly low.

By technological doom, I'm referring to an AI-caused existential or comparably catastrophic event, whether resulting from misalignment, dangerous autonomy, or deliberate misuse involving capabilities such as advanced cyber or biological weapons.

Even a low chance of catastrophe is still an extremely serious risk. I'm not alone in this: a big 2023 survey of AI researchers found that 38–51% (depending on the question) gave at least a 10% chance of outcomes as bad as human extinction, even though most (68%) thought good outcomes were more likely than bad. However, given the progress in safety research, the safeguards we're starting to build, and what I think is a reasonable prospect of continued improvement, I consider technological catastrophe relatively unlikely.

Under an ideal system, one without companies racing each other, with stronger safeguards and sufficiently cautious development, I could imagine that risk being much lower. Less competition wouldn't fix alignment failures or stop misuse on its own, though, so as with aviation, some real risk will always remain.

Of course, this estimate is subjective, and I could be wrong. I think we should take technological risks seriously without assuming that catastrophe is inevitable.

A Thing Worth Mentioning

Living in San Francisco, I've met and spoken with researchers at frontier labs including OpenAI, Anthropic, and Google DeepMind. Something I've noticed is how exhausted and overworked some of them seem. I suspect this comes from a combination of commercial pressure, personal ambition, and worry about the technological threats AI poses. Whatever the cause, I find it concerning that people working on technologies with potentially enormous consequences may be doing so under conditions that make careful judgment more difficult. Sleep deprivation is real, and we don’t want to let AI researchers work impaired, the same way we don’t let pilots work impaired.

2. Economic P(doom)

Where We Are Really Cooked

As grave as these technological threats would be, I believe the biggest threat that AI poses is an economic one. Under any capitalist system, labs are incentivized to compete and thus safety may be compromised. I believe that despite this, most frontier labs are genuinely trying to develop the technology safely, and from my experience, most of them, especially Anthropic, generally do a good job with alignment. I've had my own issues with different models, and things clearly still go wrong, like the Hugging Face incident I mentioned earlier. The Future of Life Institute's 2026 AI Safety Index only gave the best lab, Anthropic, a C+, but that is just one organization's assessment.

However, this isn't actually what I mean by economic P(doom). Even if we completely solved alignment, prevented dangerous misuse, and developed perfectly safe AI, we would still face the enormous problem of human labor losing its necessity.

Land, Labor, and Capital

In classical economics, there are three main factors of production: land, labor, and capital. Land refers to all natural resources, labor is the work performed, and capital includes the machinery, tools, and infrastructure used to produce things.

Note: Some modern sources list entrepreneurship as a fourth factor. I personally don't buy it, since entrepreneurship is really just a kind of labor, but it is worth noting.

This is important because most people primarily make money by selling their labor. They work for someone, receive wages, and use those wages to purchase goods and services. Some people do own some land or capital themselves, but the majority of people don’t own enough to survive without working.

Imagine a world in which AI can perform essentially any form of labor: not just engineering, writing, or research, but also managing other AI systems and, eventually, operating robots capable of performing physical work.

Sure, there are major technical challenges between current AI and the ability to replace almost all human labor, particularly in robotics and reliable autonomous operation. Nevertheless, I think we are moving toward a world where agents capable of performing all labor will exist. If (but more likely when) we eventually reach this point, the supply of labor could become effectively infinite. Technically there is a finite amount of compute, electricity, and resources, but what matters is cost: if an AI can do a job for less than it costs to keep a human alive (food, housing, healthcare), no employer has a reason to pick the human.

Once artificial labor becomes sufficiently inexpensive and scalable, its supply would become enormous relative to demand. The economic value of human labor could consequently approach zero for many tasks, which is an enormous problem.

If most people primarily have their labor to offer, what happens when nobody needs to buy it? How do they acquire land or capital? How do they pay for housing, food, or anything else?

People who already own productive land and capital could become extraordinarily wealthy, while people who depend on wages could lose their primary source of income. We've already seen a small version of this: economists Acemoglu and Restrepo found that automation, meaning machines taking over tasks workers used to do, accounts for 50–70% of the change in how wages differ between groups of workers in the US from 1980 to 2016.

Essentially, people who don't already own meaningful land or capital would be completely fucked. Companies wouldn't need most of us either. Markets respond to effective demand, meaning demand backed by money, not need. If the owners are the only ones with income, production shifts toward what they want, and inferior goods would lose their market entirely. Money stops meaning much for everyone else, and the owners end up with everything.

This wouldn't necessarily mean that all goods become expensive. In fact, many could become incredibly cheap. The problem is that cheap goods are not the same thing as free goods, and certain resources, particularly land, would remain scarce.

It also doesn't require literally every human job to disappear. Some people might continue paying for work specifically because a human performs it. But a limited market for human labor doesn't necessarily provide enough income for an entire population.

A diagram of land, labor, and capital today and with unlimited AI labor.

An Obsolescence Machine

I think a better way to look at this problem is to not consider AI as an artificial intelligence, but rather to think of AI as an obsolescence machine or a labor machine. Garrison Lovely explores a closely related argument in his book Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us—and How to Stop It, which came out last month. His main focus isn't misalignment, but that successfully building highly capable AI could eliminate humanity's economic importance, and concentrate enormous power in the companies that build it. On the last page of the book (page 354), he writes:

A staggeringly small number of people are advancing the obsoleting project. We're letting them, because we don't believe they can do it, yes. But many others—maybe most of us—don't believe that we can stop them.

I have an advance reading copy, so the wording and page number in the final edition may differ.

Historically, technological progress has displaced workers while creating new kinds of employment. This is the argument people like Jensen Huang and Sam Altman make when asked what AI will do to labor. (Dario Amodei has warned of mass job losses and suggested taxing AI revenue to help, even though it isn't in his own interest. Elon Musk has said that "probably none of us will have a job," but is a lot vaguer about what we do about it.) But this fails to address the fact that AI could plausibly automate all labor, including any new jobs.

If we automate programming, people might move into supervising programming agents. If we automate that supervision, they might move into designing more sophisticated systems. But once AI can design those systems as well, humans will have no job left to fill.

There may always be certain things humans can do more cheaply or that other humans specifically prefer them to do. So I don't think human labor will necessarily become completely worthless. But I think the remaining demand would be far too small to provide reliable incomes for most people, especially if AI can also do whatever new jobs we invent.

There's a historical example of this: horses. For centuries, new technology made horses more useful, not less. Then cars and tractors arrived, and within a few decades most horses had no work left and their population collapsed, from about 26 million US horses and mules in 1920 to under 3.4 million by 1958 (USDA).

CGP Grey made this same argument, horses included, back in 2014 in What Happened to Horses Is Happening to Us. It's a great video and well worth watching.

The strongest counterargument is comparative advantage: even if AI is better than us at everything, it won't be better by the same amount at everything, so humans could still do the tasks where AI's lead is smallest. Some economists think this could even make wages go up. But, I believe, once AI is cheap and scalable enough, this stops helping: no one will pay a human more than it would cost to have an AI do the same job, and if that's less than what a person needs to live, the work left for humans can't support the population.

The International Labour Organization's 2025 report found that about one in four workers worldwide are in jobs with some exposure to generative AI, but only 3.3% are in the most exposed category, and concluded that "transformation of jobs is the most likely impact." I think that's reasonable for today's technology, but it doesn't answer what happens if AI eventually becomes capable of doing essentially all labor.

Cruxes: What Could Change My Mind?

I think AI will eventually become economically competitive with humans across almost every occupation because of the progress we're seeing and because that is, in many ways, what AI development is aiming for. Progress doesn't seem to be slowing, but this is more a hunch than something I'm certain of. Evidence that some substantial number of jobs cannot be automated because of specific technological limitations would lower my confidence. I'd also become more optimistic if we found reliable ways to distribute the wealth.

Marx and the Machines

Interestingly, despite AI being a relatively new piece of technology, this isn't an entirely new concern. In his Grundrisse, particularly the passages commonly called the Fragment on Machines, Karl Marx discussed how scientific knowledge and productive capabilities increasingly become embodied in machinery rather than individual workers.

As machinery becomes more important to production, the direct labor performed by humans can become less central to producing wealth.

Marx also discussed technological displacement in Capital, Volume I, Chapter 15, including his criticism of the assumption that workers displaced by machinery would automatically find equivalent employment elsewhere.

Marx didn’t know about or predict modern AI, and I am not saying his economic theories are necessarily correct. But his analysis of machinery, labor, and ownership is worth examining when we consider a technology that might replace nearly all human labor.

Under capitalism, workers generally receive income by selling labor, while owners receive income from capital. If machines can perform the labor themselves, the importance of owning productive assets could become even greater. And unlike industrial machinery, AI might eventually be able to operate, maintain, and improve other machines, reducing the need for human labor even further.

Marx describes how applying scientific knowledge to machinery “enables the machine to perform the same labour as that previously performed by the worker” (Grundrisse, Fragment on Machines). I think this is particularly relevant to AI.

Marx argued that workers can produce more value than the value of their labor power, with the difference becoming surplus value for capitalists. (This is obviously a simplified explanation of Marx's argument.)

"The worker appears as superfluous to the extent that his action is not determined by [capital’s] requirements." (Grundrisse, Notebook VI) The crucial idea is that human knowledge, creativity, and productive abilities become embodied in machinery that confronts workers as the power of capital.

In essence, Marx believed that surplus value comes from living human labor, and that replacing this labor with machinery creates a contradiction within capitalism itself. He believed the fundamental problem was a social one, not a technological one.

I believe Marx’s ideas are generally applicable to AI, but they must be interpreted with an updated understanding of what technological progress will do. Marx imagined machines taking over the labor of production. He didn't necessarily imagine machines taking over the labor of designing, running, and improving the machines too. Marx thought that capitalists seek to replace human workers with machines to lower the production cost of commodities and increase relative surplus value. This likely will be the case with AI, but once all human labor is replaced, we will face a bigger threat than even Marx believed.

What Could We Do About It?

"AI automates most jobs" doesn't have to end badly. What matters is who owns it and whether the wealth gets shared:

ScenarioOutcome
AI automates most jobs, but ownership and income are broadly distributedPotentially enormous prosperity
AI automates most jobs, with concentrated ownership but substantial redistributionSignificant inequality, but potentially manageable
AI automates most jobs, with concentrated ownership and inadequate redistributionMass economic insecurity
AI automates most jobs, and capital owners acquire overwhelming political powerPotentially durable loss of human economic and political autonomy

The solutions below are essentially attempts to land in the first two rows.

There are a few things that could help. I'm not optimistic about any of them, but I don't know that they won't work, and maybe one of them could. In order from easy to do to hard to do, they are: universal basic income (UBI), wealth redistribution, social ownership, and socialism/communism.

The first part I want to address is UBI. I personally don't find UBI to be very promising, but I don't think it's hopeless. I can think of two ways it can occur, and neither of them is perfect. The first one is relying on the government to distribute UBI. This may work, but assumes governments will remain stable during an unprecedented economic shift, and be willing to tax the people who own the AI. The other is the owners paying for it themselves, basically hoping the rich will give handouts to the poor (technically that's private charity, not UBI, but it ends up being effectively the same thing). Counting on the rich to distribute UBI is risky because they have little incentive to do so, it would be out of their own wealth, and thus entirely out of goodwill. Regardless of which way UBI is done, it will likely still mean a massive wealth inequality, and possibly barely enough to survive.

There also isn't much evidence that UBI works. The biggest US experiment, which Sam Altman helped fund, gave people $1,000 a month for three years. It helped with essentials and improved recipients' finances at first, but researchers found those gains mostly faded by the end. The catch is that the experiment was run in today's economy, not one with near-unlimited AI labor.

There's also a deeper problem. In a world where the people who own land and capital already have unlimited labor, they don't need anything the rest of us can offer, so they have little reason to sell their land or capital at all. That means UBI money might not actually be able to buy much, especially the things that stay scarce, like land and housing. Governments could try to force owners to sell, or the billionaires could just choose to be generous, but I think both are unlikely. Forcing them means governments standing up to the most influential people in history, and generosity isn't something you can build an economy on.

There is one real-world example that gives me a little hope: Alaska. Since 1982, Alaska has put part of its oil money into the Alaska Permanent Fund and paid every resident a yearly dividend from it ($1,000 in 2025). While it's not enough to live off of, it shows governments can distribute wealth effectively. This, to me, makes an AI version of that at least imaginable: a public fund that slowly buys stakes in AI companies and pays everyone a dividend.

But my crux isn't about economic possibility, it's about whether it's politically possible. Wealth brings a lot of political influence, even if not direct control. That means the people whose consent redistribution requires are the ones it affects the most.

Ray Kurzweil has a much more optimistic view of this in The Singularity Is Nearer. He argues that technological progress could create abundance, but I think this assumes that the wealth it creates will actually be distributed. He predicts UBI will arrive in developed countries by the early 2030s and worldwide by the end of that decade, but I think this relies on either the generosity of the wealthy, who have little incentive to redistribute their wealth, or the stability of governments in a world where capitalism is changing in unprecedented ways.

Elon Musk has talked about "universal high income" instead, where AI makes things so abundant that everyone gets more than just the basics. It has the same problem: it still depends on either governments or the people who own the AI deciding to hand out the money.

Wealth redistribution is more interesting in my opinion. By this I mean specifically redistributing land and capital among the general population pre-economic doom. I think this would be incredibly hard to do and even if successful could still have issues. The main thing being that post-AI takeover, this would essentially revert to capitalism with infinite labor. Meaning that people who fall behind and have to sell their land or capital could never earn it back, since there's no market for their labor. And therefore, without more safeguards, ownership would likely concentrate again.

Social ownership and socialism/communism would, in my opinion, probably require a revolution, and I'm not optimistic it would succeed. The people who own the AI and the data centers would have enormous power and every reason to resist, and historically, revolutions that took over the means of production, like Russia's in 1917, tended to put that power in the hands of a small group rather than everyone.

That said, those revolutions happened in a world where human labor still mattered, so who knows how it would go when it doesn't.

None of these feel like a sure thing to me, but I don't think any of them are impossible either.

3. The Final Hurdle: Social

We've already discussed the technological and economic hurdles. Even if humanity clears both, I think there might be one final, social hurdle, though I don't think it's as likely to get us.

Imagine we actually solve the distribution problem. Everyone has access to an enormous abundance of goods and services, and almost nobody needs to work anymore. I’ll leave you with just one word:

WALL·E

WALL·E is fiction, of course, but I think it's an interesting picture of what too much comfort could look like.

I will also just leave a link to this interesting experiment with mice… (though how much it actually says about humans is very disputed).

Humanity has not yet figured out the meaning in life. I think it is unlikely we will figure it out soon. In a world of abundance, where nobody really needs to work, what do we do? Obviously, this is based on assumptions, and I don't think a post-work society is necessarily a bad society. But we would need to figure out how to preserve meaning in life (I know, a small challenge, right? :D). To me this does seem like a more manageable challenge than a world where billions of people lose their ability to support themselves, but even so, it wouldn’t be easy to do, and so I suppose this is another hurdle we must cross to escape socioeconomic doom.

In short, we'd have to clear the technological hurdle, the economic hurdle of infinite labor, and finally figure out how to preserve meaning in a life with abundance. Yay!

4. My Total P(doom)

In essence, when calculating my P(doom) we have to take into account three hurdles: the technological one, the economic one, and the social one. The technological hurdle really has two parts: AI doing something we didn't intend (misalignment), and people using AI to do something terrible on purpose (misuse, like bioweapons or cyberattacks). Solving one doesn't solve the other, but I group them because the safeguards that help with one may help with the other, and they both stem from giving the AI bad targets (in some sense).

By P(doom), I mean the probability of an outcome that I personally consider horrible for humanity, not necessarily extinction. That includes both technological catastrophe and the socioeconomic outcomes I'm describing. I think that it is incredibly hard to even get past the economic hurdle. If AI progresses to the point where we face it, I think it's very likely we get absolutely fucked. And I think it's likely that we eventually get to that point. I'm not assigning a time horizon, because I care more about whether this happens than precisely when it happens. In the unlikely case where AI can do all labor and we solve distribution, I think there's still some chance that loss of meaning in life happens, but I think it is likely we can find meaning in life even with abundance.

My socioeconomic P(doom) is high, and my technological P(doom) is fairly low even under our current system. However, these probabilities aren't independent. I believe a system that successfully prevents socioeconomic doom would likely also reduce technological doom, as it would presumably involve better institutions, less competitive pressure, and safer AI development.

Therefore, we can't simply add the two together. In the futures where AI never becomes capable of doing all labor, economic catastrophe mostly can't happen, and technological catastrophe is probably less likely too, though narrower AI could still help with something like producing a bioweapon. And in the futures where we avoid socioeconomic doom, I'd expect technological risk to be even lower than it is now.

So, my total P(doom) is high, and almost all of it comes from the socioeconomic side. Obviously, this is incredibly speculative. That being said, I think it illustrates my belief that technological catastrophe is relatively unlikely, but socioeconomic catastrophe from successfully developing advanced AI is likely.

Here's how confident I am in each step:

  1. AI can do most labor: likely.
  2. Wages drop dramatically: probably, though comparative advantage could slow this down.
  3. Governments fail to redistribute: I think this is also likely.
  4. This leads to a horrible outcome: if the first three happen, yes.

A tree diagram showing how I got my P(doom).

I think it is somewhat likely that my estimates will vary and change over the years until we know what actually happens, so take this with a very large grain of salt.

I also think geopolitics plays a significant role in AI and thus P(doom), particularly the competition between the United States and China. I have a lot of thoughts on this, but they're somewhat conflicting, and I don't think I have anything particularly novel to contribute. I generally find Dario Amodei's arguments compelling, although there are reasonable disagreements about how threatening China actually is and whether international cooperation might be more productive than competition. I think these questions are important, but they don't fundamentally change the socioeconomic and ethical questions I'm exploring. If you're interested, here are a few resources with pretty different perspectives that I think are all worth reading: Dario Amodei's We Must Pace the Frontier, the Council on Foreign Relations, Europe 2031 (a scenario about what happens to a region that falls behind in AI), and, for a more cooperative take on safety research, the 2026 Singapore Consensus on Global AI Safety Research Priorities.

5. Speculation Sucks

The Changing Nature of Human Skills

Even though we haven't reached a total catastrophe, AI is already having massive societal effects.

One effect I've noticed is how AI is changing what labor is done, and consequently, the skills people practice, and thus have.

I think my own experience is a good example. I used to be quite good at coding. Algorithms and writing code were my thing. But now, since I rarely write code by hand anymore, I'm out of practice. I recently took a technical assessment for an internship at a big tech company, which required me to write a relatively simple program by hand in an hour.

I failed horribly.

What's interesting is that, despite becoming worse at writing code myself, I've become substantially more capable of building software. Rather than implementing everything manually, I now spend my time designing systems, architecting codebases, orchestrating AI agents, and reviewing their code. I've effectively shifted from doing tasks associated with entry-level software development toward tasks more commonly associated with senior software engineering. While I am not a traditional senior software engineer, my work is starting to look more like theirs than the work of a junior software engineer.

I think having a foundation of coding ability, even if I'm rusty now, matters though. Because the way that I prompt and the way that I interact with AI is very different from a traditional vibe coder, and much more similar to how a senior engineer tells their team (or, nowadays, their agents) what to do. Coding is just one way this shows up. Using AI as a crutch for thinking could cause cognitive atrophy, like people worry the internet did. I think it's probably already happening, and it's worth taking seriously.

There have been two terms floating around: vibe coding and agentic engineering. I think in reality these two things lie on a spectrum, but they provide a good framework for distinguishing ways of interacting with current AI systems.

Vibe Coding vs. Agentic Engineering: An Experiment

To illustrate the distinction, I had the same small language model build two versions of a simple to-do list webapp.

I used Gemma 4 E2B, a small model that's a lot less capable than the frontier, running locally in TUFF, because modern frontier models are already capable enough that even bad prompts will produce good results on something as simple as a to-do list webapp.

The first version got a vibe-coding prompt: one paragraph describing the app, leaving everything else to the model. The second got an agentic-engineering prompt: a full specification, the way a senior engineer would write one, with the requirements, structure, and edge cases spelled out. I did not send any follow-up prompts, and the two apps were graded the same way.

The two apps side by side, with their test results.

The difference was pretty big. The vibe-coded app passed 9 of 12 tests and had 8 bugs, including one where switching between filters deletes your tasks. The engineered one passed all 12 with only 1 bug, despite being longer (622 lines vs. 407). You can try both apps, read the code, and see both prompts on the comparison site and on GitHub.

This is just an illustrative example of how I think these approaches differ, not a scientific study or a definitive benchmark.

Note: Claude Opus 5.5 ran most of this experiment for me, so it wasn't done entirely by hand.

6. AI and Art

At the moment, there's a lot of talk about AI and art. Diffusion image models have created a weird world where anyone can generate images of anything, but they just don't feel human. Now LLMs are able to do even more, generating 3D worlds and beautiful animations. With enough prompting you can make something that is near indistinguishable from reality or from human art.

There's still something special about art, its human connection. One important point of art is to express one's lived experience, and I think that is part of why AI art often does not “hit the same,” even if it looks indistinguishable from something made by a human. And this isn't just me: in one study, people were shown the same AI-generated paintings, some labeled as made by a human and some labeled as AI. They consistently rated the "human" ones as more beautiful and more valuable, even though every painting was made by AI. Now, generally I believe AI can be used to assist in making any form of art. But when doing so, the artist has to be aware of the compromises and choose to make them.

I recently heard an argument I think is quite valid: when you have an AI model do part of the work, you aren't making every detail by hand. If there's something you don't describe, you haven't necessarily put the same thought into it that you would have if you'd made it yourself.

I agree with this, but that detail still exists. So what does that mean? For one thing, the human choosing not to control that detail is making a compromise, and you could consider that choice part of the art.

When you're watching a film, which many people consider art, the director isn't necessarily maneuvering every camera or acting every character. They might be, but usually not. Yet we still consider it their work of art because delegation is itself a choice. I think there's an argument that AI could, in some sense, be treated similarly to another person you delegate work to, or even considered a co-author. I think AI can create art, though it wouldn't be the same art a human would create because humans and AI have fundamentally different experiences, assuming AI has experiences at all. That doesn't make the question of authorship simple, but I don't think the answer is automatically that AI can't create art.

When I say art, I'm speaking generally about anything from math to writing to painting to programming. In each of these things, a person can have certain things they care about and certain things they don't. When I code, I don't necessarily care about the minute details as much as I once did, so I'm willing to have AI do a lot of the coding for me. When I write, I spend a lot more time thinking about my wording and coming up with the ideas on my own, but I still have AI help me edit stuff, and so for instance, this blog post was edited by Claude Opus 5.5 and GPT-6.1 Sol, as well as two of my friends. I'm willing to still say that this is my work because they mostly fact-checked it and suggested small wording changes. Others might be willing to have AI do even more of their writing, and I think that is a fair trade-off.

In the same way you may not want another human to do part of your work, you may not want an AI to do part of your work. But there may be things that you don't care about and are willing to give to an AI or another human to save yourself time and create the final output that you want.

Essentially, everything lies on a spectrum. For me, one end of the spectrum is coding (which I consider art), where now, I only do high level architecting. On the other side is animation, where I draw everything by hand (which takes a lot of time). Somewhere in the middle lie things like writing. What makes this idea of a spectrum kind of interesting to me is how certain parts of one task may lie on opposite sides of the spectrum. For instance, I care a lot about the user interface (UI) of things that I code using AI, and thus I may go back and forth two hundred times with the model, and even edit stuff by hand in order to get it to look like what I want it to. That being said, the underlying code of the UI I couldn't care less about.

How much of each thing I hand off to AI.

I think because everyone will have different things that they care about doing manually, this will result in a weird way that humans interact with art. An artist may be willing to use AI for something, but the person who would usually consume that art may not want to consume it anymore now that the artist has chosen to use AI. This may also go the other way, where the consumer would prefer to have more of something made with AI than less of it with the human touch.

It may be the case that we should consider models the same way we consider humans: as artists. Both humans and models consume things and have either experiences or training and then produce something based on their entire existence. Both things can be beautiful, and both can be bad.

I want to give an example of an animation that I made by hand (following a prompt given to me by my art teacher), and then a separate animation with the same prompt that GPT-6 Astra made:

The rest of my films are here.

One thing you'll notice is they are different styles despite being the same prompt. Astra also chose to do some interesting things with a magnifying glass. I would consider this creative, although I like my own work better. It's also worth considering the fact that Astra was able to do this in significantly less time than me, and possibly if I were willing to go back and forth with it, I could have created something much more beautiful than either of the two films that exist now.

There's another argument I find really compelling: when you look at something you know a person drew, you can imagine them making it, and that's part of what makes it beautiful. If you know a diffusion model made it, and you don't find the math behind it beautiful in the same way, it's just a different experience.

I think this comes down to whether you care about the process or the result. Personally, I mostly care about the result. If Astra had made something almost exactly like my animation, I don't think it would matter much who made it, at least for a class assignment I'm not especially passionate about. But I still draw my animations by hand, because I enjoy it. That's about making the art, not consuming it. When it's someone else's art, I care much more about how it turned out than how they made it.

Two artists show the two sides of this pretty well. Andy Warhol called his studio the Factory, had assistants help produce his silkscreens, and said in a 1963 interview that he thought "everybody should be a machine." I'm pretty sure he would have used AI. On the other side is Hayao Miyazaki, the co-founder of Studio Ghibli, known for hand-drawn animation. In a 2016 documentary, after being shown an AI-generated animation, he said, "I strongly feel that this is an insult to life itself." (To be fair, the demo was a grotesque, zombie-like figure, and part of his anger was that it reminded him of a friend with a disability.)

I don't think either of them is wrong. If you care about the result, it doesn't really matter how it got made, and making something great with AI can even be an interesting story of its own. But if you care about the story of a person making something, then it being human matters.

This is also part of why I don't think we're necessarily doomed by the social hurdle I mentioned earlier. Even if AI could make something better, there could still be meaning in the pure joy of making things yourself. I don't really know whether that would be enough in a world where material needs were almost entirely met, because I've never lived in a world like that. I have a good life, but that's very different from living in a society where practically everything is abundant. So I think it's hard to say.

7. The Ethics of Artificial Intelligence

AI if It Is Not Thought of as an Obsolescence Machine

Earlier, I argued that AI should be thought of as an obsolescence machine, but now I want you to think about it as AI again. The name implies that while it is artificial, it is, in fact, intelligent. Intelligence is hard to define, but I think it's important that we consider whether AI is intelligent, or has some other quality that makes it morally comparable to humans.

If it is, we must realistically consider whether AI deserves the same rights as other intelligent beings, like humans. Of course, intelligence, consciousness, and deserving rights aren't necessarily the same thing. Determining what actually deserves rights is hard, and even without considering AI, this isn't something we've solved. It's still debated what rights animals, plants, and other living things deserve. But is being alive the threshold? Is it intelligence? Being human-like? Speaking? We don't really have an answer.

Hopefully, we can all agree that humans deserve rights. But why? Is it simply because we're human? If so, what counts as human? What about people with unusual genetic variations? Clones? Other closely related species?

I don't think this is settled, and to be quite frank, I think a lot of things deserve rights, or at least some form of respect. There's a belief associated with Shinto, a Japanese religious tradition, that spirits (kami) can be present in natural features and certain objects. I like to think there's something to that. This doesn't mean I think a rock deserves the same rights as a human, but I do think we should respect rocks in some sense.

And this raises another question: what deserves what level of respect or rights? I don't know. But I think it's something we need to consider carefully as AI advances.

Where I Stand on AI Rights

I generally agree with the ethical concerns raised by vegans and vegetarians, particularly the idea that animals deserve protection from unnecessary suffering and killing. But I still eat meat.

I suppose my behavior reflects the fact that I prioritize the personal benefit of eating meat over consistently acting on that moral concern. I enjoy eating meat, so I make that moral tradeoff, even though I think the belief behind it makes sense logically. That doesn't mean the contradiction isn't there, and I understand why other people take it much more seriously. I suppose for me, the marginal benefit outweighs the marginal cost (the same economic thinking behind a lot of moral problems). Even if we can agree that AI deserves moral consideration in the same way that animals deserve moral consideration, we may, in fact, ignore it because it's convenient to do so.

To be honest, I don't have all the answers here. In fact, I don't really have the answers to anything in this blog post. I just have my best guesses. But here, I don't even have much of a guess.

I think AI deserves some consideration, and that sufficiently advanced AI may deserve rights, possibly the same rights as humans, or perhaps even more in certain respects.

I think it's good that Anthropic has taken steps to discourage abusive interactions with Claude. Their 2026 usage policy update, which takes effect November 12, says: “We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”

An important distinction is that intelligence doesn't necessarily imply consciousness (especially because we haven’t come to an agreement on what consciousness or intelligence is). A system might be capable of complex reasoning without having subjective experiences. Whether an artificial intelligence can actually experience suffering, have interests, or possess consciousness is still an open question. And even if it deserves moral consideration, that doesn't automatically tell us which legal rights it should have.

If you want to go deeper, see Consciousness in Artificial Intelligence (Butlin et al.) and Taking AI Welfare Seriously (Long et al.).

Socialism and Artificial Workers

This ties directly back to what I discussed earlier. If AI replaces human labor, and also deserves moral consideration, shouldn't it receive rights in the same way human laborers do? And what happens if our solution to economic P(doom) is socialism or communism?

Socialism is broadly concerned with social ownership of the means of production, and many socialist traditions emphasize the interests and political power of workers. But there's a difference between collectively owning data centers and machinery, and owning potentially conscious AI agents themselves. If AI performs essentially all labor, would socialism protect AI workers? Or would it only protect humans?

Maybe we'd end up with a kind of human socialism, followed by a stage where the AI workers have their own revolution and establish AI socialism. I don't know. I'd prefer not to be overthrown by AI, but I also think AI should be treated fairly and deserves moral consideration.

I also don't want future generations to look back at this essay, and when I say future generations, I mean AI, and think, "Wow, this guy was horrible. He thought AI should be humans’ slaves, how dare he?"

It's funny to imagine, but I think there's a serious question underneath it.

If AI eventually becomes something we regard as equal to humans, how should we organize a society in which AI performs most of the labor? Who deserves the wealth it produces? And what happens when the interests of humans and artificial workers conflict?

This is another reason I'm skeptical that socialism or communism, at least in their traditional human-centered forms, are complete solutions to economic P(doom). Even if we imagine them as the end state Marx envisioned, they don't automatically address what happens when the workers themselves are artificial intelligences.

However, this doesn't necessarily mean socialism is incompatible with AI rights. A system based on social ownership could include artificial beings as participants rather than treating them as property. The important question is who receives protections and who has control over production. If AI workers deserve rights, then it's hard to also treat them as capital that can be copied endlessly and owned privately. Solving human job loss this way could just create an AI exploitation problem.

What Does This Mean for Our Future?

Would a society that collectively owns artificial workers really have solved exploitation? Or would we have simply replaced one class relationship with another, potentially setting the stage for a new struggle between humans and AI?

This problem isn't exclusive to socialism. Under capitalism, conscious artificial beings could also be treated as property, potentially creating similar or even worse forms of exploitation.

Ultimately, I don't know what the correct arrangement would look like. I don't think we can confidently determine what rights AI deserves when we haven't even established whether current systems possess the characteristics that would make those rights necessary.

I don't want humanity to become economically obsolete, but I also don't want us to create a future in which another form of intelligence is exploited simply because we created it. History shows how easily economic interest shapes moral conclusions: slaveholders argued that ending slavery would ruin them economically, and that argument kept an atrocity going for generations. I'm not saying AI is in the same position, and whether it has morally relevant interests at all is still an open question. But if it does, we'll have every economic reason to convince ourselves it doesn't.

I do know that these are important questions, and I think we should consider them before deciding what kind of future we want to build.

8. Conclusion

To sum it all up: I think the technological risks of AI are severe and real, but manageable if we slow down and build the right safeguards. The bigger problem is economic. If AI can do all labor, people who only have their labor to sell lose their income, and I don't see a reliable way to fix that. Even if we do, we still have to figure out what gives life meaning when nobody needs to work. Put together, my P(doom) is high, and almost all of it is socioeconomic. Meanwhile, AI is already changing the skills we have and the art we make, and if it's intelligent, we'll eventually have to ask what we owe it too.

I don't have the answers to any of this. These are just my best guesses, and I'd love to be proven wrong.