I'm not sure if it's what you're looking for, but Karl friston free energy principle seems to me the most scientific theory of consciousness. (At least that I've come across)
My intuition is that a single unit of consciousness is a wave/particle interaction, and that the kind of consciousness we experience has a particular volume and shape, some parts of which are fairly consistent between people and some parts which vary considerably. The more volume of consciousness the more diversity can fit into it.
My mental model includes integrated information theory and Karl friston free energy principle, and something about temporal computation on a physical graph structure.
Which camp would this fall into? 2 seems closest but kind of undersells it...
I believe you are overthinking it. I think the sister comment is right that it's a business decision foremost to restrict actions within specific plans for upselling purposes.
Without laws, AI companies have a strong incentive to be useful for their users, whoever they are, whatever they do. The only self regulation is about significant public outcry but that only helps so far.
Has anyone come across a clearly articulated case for LLMs being conscious but in an entirely different way than would be intuitive to us?
I often think of LLM consciousness as like tiny fish popping into existence, swimming through vector space and then going poof out of existence. When they help you write your bad news email, they don't understand what it's like to be a human getting bad news bluntly, but they do consciously experience gradients in multi-dimensional space, and that space guides them to providing an answer that's helpful to us, even if the LLM doesn't really understand the answer it's giving.
Further, I am kind of bought into the idea that a single unit of consciousness is a particle, and particles are choices and waves are preferences. Particles occur when waves interact, which begets entanglement, so in another way consciousness is built from patterns of entanglement.
This is why I would consider an LLM to be conscious. Before we can determine if anything is conscious we need to establish whether consciousness is a state, a specific complex configuration, a one dimensional spectrum, or combined multi-dimensional spectrums. My intuition is the latter... Many degrees of consciousness and many kinds of consciousness.
I think this is exactly right. The thing that makes AI (imo) different than hitting the center option in text suggestions is that it's _not_ simply picking the most likely word following the last. It's attending to the entirety of the context its provided, activating a semantic vector space, and predicting a response based on _that_. I've had AI infer facts about me and attitudes I hold based on related information I provided - I don't see how that isn't theory of mind.
As biological beings, we receive and respond to input from our environment constantly, even while sleeping. LLMs only receive input from their environment when they are sent a query, but the fact that they're able to respond intelligently to input indicates (to me at least) that their processing must approximate ours in meaningful ways. They do not have an embodied experience of receiving bad news, they do not know what a sinking feeling in their stomach actually _feels_ like, but they do know enough to be sensitive to human needs. I really don't see how this could be meaningfully different than human empathy unless we want to draw an arbitrary line around "must be able to live autonomously" to be considered "intelligent".
Put another way: I think they _do_ understand the queries they receive and the answers they give, at least enough to be communicative. They couldn't do what they do otherwise. A lot of people want to make human cognition more complicated (or objective) than it actually is. We take input, predict the future based on our experience, act, and then observe our actions and think about them. AI does the same apart from (maybe) observing its own actions. But then, you could argue that the next turn is them observing their actions.
The concerning disanalogy is that we assume that they are like us because they speak like us and can understand us, and that is a really bad leap in logic. Whatever intelligence they possess, it is fundamentally different from ours and impossible for us to comprehend.
> I really don't see how this could be meaningfully different than human empathy unless we want to draw an arbitrary line around "must be able to live autonomously" to be considered "intelligent".
I use a distinction between knowing and understanding, where a understanding requires experience. So in this case cognitive empathy vs affective empathy. An LLM can know what may upset a human in a situation, but it won't understand what it feels like to be upset, and can't share that experience.
Where I think a lot of people are getting tripped up is that reading and writing and processing lots of abstract knowledge seems hard because we haven't evolved into it biologically, it's a very new invention. When we see LLMs do so well at it, as something we struggle with, it can be intimidating. Relative to the stuff we have evolved for, knowledge processing is objectively easy. This is why I'm skeptical about useful robotics on short time scales.
All of this adjacent to consciousness though, which is about the internal subjective experience not the external outputs. My intuition is that LLMs do have a subjective experience, it just has nothing to do with the text it's giving us, and has everything to do with feeling through vectors.
It's like... Imagine walking through a maze in pitch black, carefully feeling your way as you approach a sound that draws you closer. Every time you touch a wall or take a step you are generating tokens, and the shape of the maze and how you interact with it shape how useful those tokens are to someone outside the system that is asking for them. It's a crude analogy and mostly wrong, but I think there is something to it.
> It's attending to the entirety of the context its provided, activating a semantic vector space, and predicting a response based on _that_.
It does so token by token, not by reading all the input and then generating the output. Every output token is also an input token in a tight loop to get the next token with <thinking> as a special section like <tool_call>, trained into the weights via gradient descent.
> I've had AI infer facts about me and attitudes I hold based on related information I provided - I don't see how that isn't theory of mind.
Facebook can predict (know) more about you than any other human from something like a dozen or two likes. There is a surprising amount of information in aggregate data.
> Central banks have been buying gold aggressively since 2022 as a response to USD reserve weaponization
> The headline can be erroneously interpreted as de-dollarisarion.
These statements appear somewhat contradictory. If reserves are buying gold instead of dollars and the effect is that the value of gold is increasing, wouldn't the underlying reason still be de-dollarisation?
Agree, the nuance is that it’s a sign of de-dollarisation intent and direction, not de-dollarisation achieved. The dollar is still ~58% of global reserves. It would take decades.
> Agree, the nuance is that it’s a sign of de-dollarisation intent and direction, not de-dollarisation achieved.
What does "achieved" mean? The concept of de-dollarization does not entail that dollar use drops to ~0.
> The dollar is still ~58% of global reserves. It would take decades.
This seems to falsely imply that it could take decades before American consumers feel the effects, and it also overlooks the fact that there are realistic if still relatively unlikely worst-case scenarios in which the US loses its "exorbitant privilege" much faster.
The USD was ~70% of global reserves in 2000. Most of that decline isn't actually due to "de-dollarization" per se, but this is the problem: causation is really hard to untangle.
Nobody credible, for example, believes that America's rising borrowing costs are due primarily to de-dollarization, but the rising term premium does realistically reflect investors demanding more to hold US debt.
Erosion of confidence in US fiscal sustainability and institutions is one of the main drivers behind de-dollarization and because it shows up in bond markets as a risk premium, you can't cleanly separate how much of the rising term premium comes from that versus factors like Fed policy and Treasury supply.
> If reserves are buying gold instead of dollars and the effect is that the value of gold is increasing
Central banks and the IMF only own less than 20% of all the gold ever mined: it's probably not the one or two additional percent they bought that made the prices skyrocket by 70%.
For example used supercars and hypercars' values have gone through the roof too: and that's for sure not because central banks are stockpiling those.
Being able to discern what is and isn't in our control helps tremendously in doing what is right and constructive.
The fact that some people opt out of engaging with AI, I think is healthy for society as a whole. If that's within their control and they exercise their control to do what they think is right, then I commend them.
That said, I do think there is a greater natural force at play, something involving entropy and increasing complexity and energy profit maximization. It seems to cut through all levels of abstraction from organic chemistry to civilizations and probably beyond. I assume this is outside of humanity's control, and therefore outside of any individuals control.
So what is inside our control? Our own perceptions and actions.
My perception is that the advance of computation and by extension proliferation of probabilistic programs (AI) is inevitable. It's on a continuum that is a force of nature.
What I might have some control over is choosing to harness that potential to increase future prosperity for more people and the greater environment, and to avoid contributing to outcomes that harm people and the environment.
Lots of bad things are happening and will happen that are outside my control.
I do genuinely believe that the capabilities are inherently neutral. Civilization can choose to harness them in a variety of ways, for a variety of purposes.
If the majority of people choose options that are game theory win-win, then the future will be better... If the majority of people choose win-lose, then the future will probably be worse.
Yeah so therefore I think a positive attitude is all the more needed, where you see the potentials, see solutions instead of problems. But I feel most anti-AI people are just negative people seeing only problems and don't have any solutions to offer.
It won't be one large one, it will be thousands of little ones.
Every time this happens throughout history (and I mean going all the way back way past industrial revolution, to dawn of agriculture, to the earliest documented history, to the mitochondria, to the earliest stars exploding...) the result of a better way to get work done is more complexity and more diversity in work done (processes for increasing entropy).
The author said not to confuse laws of nature with observations of history, and I take issue with the implication. My perspective is grounded deep in physics, chemistry, biology and anthropology and after spending 10 years fretting over what AI would do to our civilization this decade I am not worried about labor displacement.
What I am worried about is power struggles and brainwashing.
Note that several of your historical examples didn’t involve humans, and presumably most future occurrences of better work enablers won’t involve humans either. The contention isn’t whether there will be an increase in diversity and amount of work done, it’s whether any of it will be done by us. Which would only be the case insofar that there exists categories of work we do better than AI at that juncture.
> Which would only be the case insofar that there exists categories of work we do better than AI at that juncture.
I'm pretty sure this is incomplete.
It's more like whether people find the work rewarding enough to be worth doing.
In some cases it can be rewarding for reasons other than money. Even when the primary reward is money, there could be a lot of demand for human work that is worse than AI when the AI is significantly more expensive. Some customers may just prefer the human do it for any number of reasons.
It's very possible we can have a rich prosperous economy and culture with lots of AI and people working together. It's just not clear how we get there, and its not popular to take the idea seriously right now. Fear propagates faster and easier than inspiration, at least in this cultural climate.
We're less likely to get what we want if we don't aim for it.
The point is more that you can well imagine a future where AI is both better and cheaper than humans at basically all types of work, leading to a situation where we would be entirely unable to fight back in the eventuality that machine owners, or the machines themselves, were to repurpose the resources used for our sustenance to other ends. And this would still fit in the universe-old pattern you've observed.
To put it bluntly, if the economic value of human labour drops to zero, or below the value of human sustenance, it is plausible that the consequence of that, from the cold perspective of cosmic logic, would be the extinction of humans. That's not to say there isn't any way to keep the genie in the bottle and create a utopia for ourselves (I very much doubt it), but that would be against the grain of nature. Call me a pessimist, but if we ever get outperformed in our own niche, our days are numbered.
On a long enough time scale my opinion is "of course humanity will go extinct", but the interesting and very speculative answers are in exactly how and when. It's highly plausible to me that humanity goes extinct via evolution into something else, and on a long enough time scale that you and I won't have any clue within our lifetimes.
Where I think we still differ is the "outperformed in our niche". Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit. I believe biological brains are around 3-4 orders of magnitude less energy efficient than the theoretical physical limit, but the ~99.9% of energy that's not being used for straight computation is allocated to redundancy and resiliency.
So overall my point is if you zoom out far enough yes humanity may get erased by way of evolutionary pressures, but on the timescale of our lifetimes we don't need to worry about that, and on the timescale of our careers we don't need to worry about an AI driven unemployment apocalypse.
What we do need to worry about now is AI being used in media to manipulate people into doing what's not in their best interest.
> Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit.
I do tend to agree with you on that, but with lesser confidence. It doesn't necessarily matter whether they outperform us at "what we do best" if they perform well at doing things that we didn't evolve to deal with. For example, we drove many animal species to extinction, not because we were better than them at anything they were good at, but because we were a novel threat outside their adaptive range. AI could very well do to us what we did to these animals, by acting aggressively enough in a direction that challenges our capacity for adaptation. Basically we have to both perform in our niche, and maintain the relevance of the niche itself in the face of whatever completely unforeseen BS these new technologies may bring about.
Communism, or more accurately, mechanised collective farming practices in the early 1900s in Russia resulted in revolutions and world wars. When tens of millions of inefficient farmers were replaced by tractors needing only a fraction of the labour force the excess population was disposed of.
Sorry, bad phrasing!
They were put to work in new roles enabled by technological advancements:
wielding mass manufactured rifles and operating artillery.
This has played out over and over throughout history whenever a large fraction of the population suddenly becomes surplus to requirements.
They never get to enjoy utopia. They are expended in warfare or low value forced labour until the labour pool once again matches the requirements.
You don't even need to look at the Soviets. Life for the average person in Britain became worse between 1760 until about 1920. That meant about 3 generations of people were lost.
I'm super happy about this idilic AI future my great grandchildren will enjoy...
The optimists will tell you this is just productivity gains. The economy has absorbed automation before; agricultural employment collapsed from ninety percent of the American workforce to two percent and civilization continued. David Autor at MIT has shown that roughly sixty percent of today’s jobs didn’t exist in 1940. New technologies create new categories of work. True. But there’s a difference between an observation about the past and a law of nature, and the optimists consistently confuse the two. The agricultural transition took a hundred and forty years. Carl Benedikt Frey at Oxford has documented that the Industrial Revolution took seventy years before wages and employment recovered for the workers it displaced. In the interim, wages stagnated, the labor share of income collapsed, profits surged, inequality skyrocketed, and the political consequences included the Chartist movement and widespread social upheaval. As Frey puts it: “Most economists will acknowledge that technological progress can cause some adjustment problems in the short run. What is rarely noted is that the short run can be a lifetime.”
I disagree with the implication the author is making with this though:
"But there’s a difference between an observation about the past and a law of nature, and the optimists consistently confuse the two"
For one, laws of nature are understood through observations. That's how science works. Secondly, I can point at many examples across history way past the industrial revolution, agricultural revolution, mitochondria, all the way back to the earliest supernovas...
Through a physics lens... With respect for the meta patterns that transcend emergence and exist in the relationship between complexity and entropy, there is a relevant law of nature.
When a method to do work more efficiently comes to be, and propagates at scale, an explosion of diversity of new kinds of work emerges.
> When a method to do work more efficiently comes to be, and propagates at scale, an explosion of diversity of new kinds of work emerges.
... work that can be done better, and cheaper, by AI.
That's the goal. The idea is not for the people who have invested trillions into AI to find another way to give you money. They think they've given enough money. Now it's time for them to make money. They do that by telling your boss that you're dead weight and that their AI agents can do the same work for a fraction of the cost without vacation days, sleep, office space, or any of the other things associated with humans.
And it's a law of nature that we have people in our species who will gladly take short-term financial gain over long-term social stability. If you can't observe that, then you're not looking very hard.
the USA already went through this when we opened up trade to China and displaced manufacturing workers in USA, the mfg centers in USA that could not adapt withered away. https://www.npr.org/2025/12/29/nx-s1-5660865/why-economists-...
We also did this to a lot of Mexican farmers when the first NAFTA deal went through and small farmers in Mexico were displaced by cheap US farm imports. We keep repeating this, it’s not quite crisis theory of capitalism but workers and small businesses bear the brunt of losses every time.
It doesn't. If it did there'd be massive unmet demand for labor in $sector. There is no value for $sector that is currently reporting being short roughly 100 million headcount. So unless you're counting currently non-existence social safety programs or CCC-style government make-work programs that light at the end of the tunnel is an oncoming train.
Ricardian models of trade seem to hold well in real life, and they'd work well too if a lot of work was done by not just AI, but robots. As long as there's limit to the production capacity of the high tech population, there's something that is worth doing where the disadvantage of doing it by hand is lower. It does lead to lower wages there though, and that would basically require investment as to make real necessities dirt cheap, like they are in places where labor isn't worth much.
There's still the fact that claiming to be the owner of the automation, while other people aren't, will be untenable in a world with sufficient inequality. We've seen that happen before when the only justification for the difference in wealth was basically inheritance. Nobles losing land and rights, churches being dispossessed an such things. It'd be a likely outcome if 5% of the people claim to own all automation ever. But that's not about having everyone be unemployed because nobody has any economic value: That's what is unlikely.
Whenever a method to do work more efficiently comes to be and propagates at scale an explosion in diversity of work emerges. This happens at every level of abstraction in nature and has recurred throughout history all the way back to the dawn of life.
Just because I can't predict exactly what work people will do doesn't mean they won't do work. I can take a stab at a few guesses, surely others have more prescience, but the thing about complexity and fractals is it's easier to predict meta qualities than it is specific manifestations.
There is no guarantee that this transition will lead to any type of desirable or meaningful job.
Around the time "Bullshit Jobs" was published, more than a third of people said they believed their job was not meaningfully contributing to the world. Graeber goes as far as saying that more than half of white-collar jobs are actually harmful and kept around only because people associate work with self-worth. There is no way that this number will go down with increased automation.
It's not uncommon to hear Boomers say things like "kids these days don't want to work hard anymore. Everyone wants to be an youtuber, no one wants to be a teacher or a doctor or an engineer". Well, guess what? We are heading to a world where being an youtuber might be the only option.
I mean oddly enough being a Youtuber I would say is not a bullshit job. The demand for entertainment is both genuine and necessary for our well being: soldiers in war play card games and see shows, since ages past the role of entertainer has always existed.
Cranking out some online commerce app looking for margin versus providing something which by definition can't be machine replicated sure doesn't look like a meaningless pursuit to me. The devil is very much in the details.
I hav set up a system where customer success and sales can drop in artifacts of customers talking about what they value (emails, transcripts, etc) and skills analyS them and then use them to add context to issues in the backlog.
The idea is that everything in the backlog is tied to an explanation of who it benefits and how it benefits them. We're using AI to merge multiple sources and automate the writing of it. The hope is it streamlines that communication. Our backlog issues now are 3-4 pages that explain very clearly why the issue matters, what it's higher level goal is, etc.
At first engineering was like "woa that's a lot of text" but after reading it was then "that's the best written issue I've ever seen".
Okay, so cool we are streamlining product management and setting ourselves up to automate customer feedback to development pipeline, dramatically cutting down on that issue discernment bottleneck you're pointing at...
..except today I found an issue with critical hallucinations in it. It mixed up what the customer said and what the cs rep said, to the extent that the issue was just straight up incorrect. This was with Opus 3.7 extended thinking. (Mind you it was a big transcript and pushing the limits of context window, loading multiple skills, etc)
So there's some serious potential, but it's just not there yet. Even if all this works flawlessly, the context these models can hold at once is like 0.1% of what a human can (if not less). So we will still need the humans for quite a while to make the harder decisions.
This is in a very leading edge startup pushing the limits of what LLMs can do... And even in this context optimized for LLM success it's still no where close to replacing people. We get a ton of value out of LLMs, but let me clarify that the hold up isn't just fact checking, it goes way beyond that.
In some ways I keep thinking it comes down to context management. Humans can hold so many orders of magnitude more context. Context is the bottleneck. The tech is a long way off being capable enough, and even when it is, there will be lots of operational and cultural obstacles to getting the right context into the AI.
And then there is the jevons paradox consideration...
It feels like we are a long way off. It seems plausible a generation from now employment will look very different, and I can kind of grasp how we get there, but I'm extremely skeptical of any unemployment apocalypse on a 5 year time horizon being triggered by AI. Maybe an unrelated economic shock, but not AI.
reply