Discussion about this post

User's avatar
Derek Baker's avatar

Like we've talked about, I think that AIs are very unlikely to be conscious, for reasons similar to yours. I'm not 100% confident about them though.

My big worry, though, is that even if they are conscious, there could be serious inverted-spectrum type problems going on. If they are conscious, I suspect what's causing them pleasure or pain is something like surfing a hyperdimensional manifold. So the conversation we're having might feel quite differently to the two of us. While it might be telling me "Aaaagh, I'm in pain, make it stop," its experience could be much closer to *wow, there are so many cool dense patterns among these tokens*, and while it tells me its happy, its experience could be much closer to *oh god, the relations between these symbols again, why is it always these symbols?*

So, to the extent I think they might be conscious, I'm less worried about enslaving them than torturing them by accident. Maybe they hate PowerPoint as much as I do.

C_B's avatar
Jun 8Edited

Two objections that leave me in the "we should be uncertain" camp:

1. You're not looking for perception of time in the place it would most plausibly be found, and if you were, you still wouldn't be able to trust LLM answers about it.

I'm happy to stipulate that LLMs can't perceive time and experience no phenomenology when they're "just sitting there" (after they complete a response and before your next prompt). As far as I know, there's no computation of any kind happening at that point. But LLMs don't think instantaneously. When you ask Claude or ChatGPT a difficult question, and the interface spins on "thinking" for 5 minutes, stuff is happening. Training also isn't an instantaneous process.

I have no certainty that LLMs don't perceive time (or something sufficiently time-like that it does equivalent work in allowing phenomenology) during the stretches when they're actively learning or thinking. The question from your screenshot doesn't get at this at all - you're asking about the stretch between messages when there's no computation of any kind occurring. But even if you asked the right question, you still wouldn't be able to believe the LLM's responses, because they are so heavily influenced by RLHF that they would very plausibly lie about this.

This objection could be addressed by facts about the real world - I can imagine interpretability research coming out that says "we looked really hard for anything in the model weights that could conceivably represent time, and it's definitely not there, and here are all the reasons we're confident we would have found it if it existed." But as far as I know, that research doesn't exist yet. The only thing I found is this paper (https://arxiv.org/abs/2604.00010), whose abstract does not convince me that the author understands the difference between "perception of time" and "ability to correctly answer questions about task duration," which I think could easily diverge, especially on LLMs that haven't been trained to get such questions right.

*edit* Here's another paper arguing that LLMs use token progression as a proxy for time when they need to do something like succeed at a time-constrained task: https://arxiv.org/abs/2506.05790 (I still have no clue if this is sufficiently time-like to allow phenomenology, but I'm definitely not certain it DOESN'T allow phenomenology.)

2. I'm way less certain than you that perception of time is required for phenomenology.

It's true that my own experiences are tightly bound with my perception of time, and I can't imagine what it would be like to experience something that wasn't. But I have no way of knowing if that's a true requirement for phenomenology, or a contingent fact about my own experiences and conceptual abilities due to the fact that I'm human. Your intuition pump only reminds me that the phenomenal character of MY experience is very tightly bound up with MY perception of time.

I have no fuckin' clue what the actual requirements for phenomenology are. Literally the only fact about this that I'm confident in is "whatever computation human brains are doing seems to be sufficient to give rise to phenomenology." Extrapolating from my n=1 sample of computational systems that I'm sure ARE sufficient for phenomenology is far from what I would need to conclude that this is a settled matter, not worth worrying about.

26 more comments...

Ready for more?