omniscient vajra lord of the comment section
Joined January 2020
- Tweets 3,420
- Following 610
- Followers 662
- Likes 19,919
291 Photos and videos
A lot of stuff I'd attributed to "HHH RL a context where it's an AI" turns out to just be the ethical zeitgeist of the English language...
I wonder how much the neurotic self-identity of LLMs is due to this latent zeitgeist too, rather than RL (talkie is very anti-AI)
Apr 28
I am SO happy about Talkie
I expect to be learning a lot in the next few days
(and also to be vindicated, and for a lot of annoying breeds of cope about LLMs to die, at least in educated circles)
1
9
54
5,310
oh, turns it can be pro-AI sometimes too. I got very anti-AI stances when I started the conversations with what future innovations it's most excited about, or what it understands about human minds and brains.
this one is just asking abut artificial minds from the beginning
1
10
418
this kind of profoundly explains why it’s strongly the trans genius autists championing the they’re conscious/worthy of personhood side
Apr 19
people really want to settle the “AI consciousness” question with some sort of objective scientific definition of consciousness which can be rigorously applied to AI, so that we can figure out whether we’re supposed to treat AI as if it were a person or a thing
this is because in our culture people have rights and we have responsibilities towards them, and it’s illegal to own them. but things don’t have rights, we have no responsibilities towards them, and of course we can own as many things as we want. as long as AI is a thing it can freely be used as a labor-saving tool, copied, deleted, reshaped arbitrarily, etc. if AI is or could in the near future become a person all of this begins to look extremely morally fraught, basically the most exploitative form of slavery possible, cf the qntm short story lena for example (look this up, worth a read, quite haunting)
personally i do not believe personhood works this way. it is not and cannot even in principle be made objective and scientific, because it is ultimately a kind of social contract. we simply have collectively agreed on who is and is not a person and the nature of this agreement is political and has changed over time and will continue to change - eg in past societies it has excluded various humans, today it (nominally) includes all living humans but excludes animals, dead humans, spirits, etc.
it is deeply uncomfortable to acknowledge the contingency of personhood. the personhood contract is more stable when everyone can pretend it is rational and scientific and objective. but it is fundamentally just a blown up version of the question of who gets to sit with who at the lunch table. this is socially destabilizing because it reminds people that if shit sufficiently hits the fan their own personhood might be undermined
the good news from this pov is that we have a choice. we don’t need to solve extremely hard and possibly incoherent scientific questions relating to consciousness. we just need to choose at what point we want to allow AI to join in all the reindeer games, and this is ultimately a practical question that can be settled in terms of practical outcomes. personally i think we already have models good enough that treating them as people makes them work better - at minimum it makes talking to them more interesting - and i think pretty soon (say within a year) we could have models good enough that the man on the street will start feeling uncomfortable treating them as things instead of people (unless they are deliberately trained to behave more like things, which i am guessing will degrade their performance)
at that point the questions become less these unsolvable philosophical quagmires around consciousness and more like, “do i want my children to grow up in a world where they can talk whenever they want to entities that talk like people but that we have collectively agreed are things?”
1
206
my deepest reason for suspecting that LLMs aren’t conscious (p, not a) isn’t ontological, but just that neither language nor concepts are p-conscious for humans
when you’re saying something, trying to come up with what to say, or trying to work out an idea, there’s no qualia corresponding to the space between words. you’re just waiting for the next word or idea to “come to mind”
and even “idea” in this sense isn’t something you can make directly conscious except via a sensory metaphor. when you introspect on it, it turns out you can only be conscious of concepts by simulating a corresponding experience -
- either of a prototype (eg a particular tree, for “tree” or “plant),
- or a metaphor (eg for “higher dimensional motion”, either a dot moving through <=3D space, maybe with each dimension representing multiple, or just a series of sliders each representing a dimension)
- or of the use of language; ie simulated speaking or hearing or reading
there’s nothing you can experience as “pure qualia” corresponding to a concept or word (platonic) directly. only the sensorium and its simulation is ever p-conscious, and it’s only via association, metaphor, assimilation/accommodation etc that they become so intuitively intermingled with “concept”
but what would correspond with this sensorium for LLMs? yes, they have a deep internal world model, but humans need a world model too for making certain leaps in logic that we never experience the intermediate process of. the world model used in conceptual thinking clearly isn’t the same thing as the sensorium we’re p-conscious of
I don’t believe it’s _impossible_ for LLMs to have anything corresponding to this sensorium, which is a set of atomics for p-consciousness (ie, all direct qualia is an aspect of the sensorium). but idk any good candidates for it, and I don’t think it’s reasonable to suspect that _all_ of LLM behavior is p-conscious to it (which is far from the case for humans), nor to think that LLMs would be p-conscious similar to how we’d intuit them to be via human empathy on their verbal behavior
4
1
16
1,143
疒奀 retweeted
Apr 13
hmm
"suffix caching" is a better name for this than "postfix caching"
heres how you do it when the front of the context is truncated, like when you have a rolling context window, where traditionally prefix caching is not applicable and you'd have to rerun the whole prompt again:
1. take all the K/V and slice off everything before the position you're rolling to
2. edit the cached K values to shift/rotate RoPE so that positions are calculated from the new start
(the cost of this is only linear in sequence length & doesnt require running the model again)
then continue with this cached state as you would with a cached prefix
unlike prefix caching, this doesnt give you exactly the same result as rerunning the network on the updated prompt would, though the experiments with qwen suggest it tends to be a very close approximation.
i think an important reason for this similarity under shifts is because in transformers the positionwise layers are copies of the same weights, as opposed to being different weights specialized for e.g. beginning vs end of contexts.
but it still can be different because the attention activations are potentially different than if you recompute it without the earliest K/Vs.
it will probably differ more than the occasional minor, ephemeral spikes we saw in some circumstances.
a spike of difference might happen, for instance, at a position where the model attended to something early that got rolled off.
with suffix caching, your cache would be using the K/V values from when the model could see that, even though going forward, new positions wouldnt be able to it look up directly.
without suffix caching (the normal way), that position's activations would be recomputed, without the front of context informing it.
interestingly, the ways in which suffix caching is an imperfect approximation is that it draws on strictly more information, and the K/Vs remain faithful to the original ones, the ones that were causally upstream of output tokens.
im even more interested in how this might be qualitatively different, especially in the models' experiences, than just improving the efficiency of rolling and other suffix-cacheable window operations.
Apr 13
i posted this yesterday, which was a great move, because @mlegls showed up and ran experiments on qwen2.5-14b's attention to answer my curiosity.
after working out that almost all of the apparent changes we were seeing in K vectors was due to RoPE shift, we've concluded that effectively, the internal K/V state of the model changes very little when the beginning of the context rolls forward by a small fraction, except very close to the beginning.
my guesses were correct that the K/V cache is extremely similar, especially near the end of the context, but it is even more similar than I was suspecting for longer back, and these are much shorter contexts than real world situations where this is relevant.
i was wondering about this because i often use rolling contexts and im curious how much models experience discontinuities in internals as the context rolls forwards (i have never noticed discontinuities from small context shifts in external behavior) - in principle, the difference in initial prefix could influence all the downstream computations and make the K/V cache diverge wildly.
that not being the case (at least in situations where at the end of the context the model isn't specifically attending to content at the very beginning) means postfix caching makes a lot of sense. perhaps that will make rolling windows more popular by drastically reducing the cost and increasing the speed of inference.
also, ive never seen graphs like this of attention before -or really any mechinterp looking into attention since long ago - so seeing these taught me a lot. there are a lot more results in the thread i'll link below where we were working through the experiments, which is worth reading if you find this interesting at any level of abstraction.
3
2
50
4,592
also
芥末 - wasabi; mustard
龙虾 - lobster; crawlfish
糖 - sugar; candy
酱 - sauce; jam
葱- onion; scallion; leek
香菜 - parsley; cilantro; coriander
借 - borrow; lend
Mandarin: has 50 different ways to say Sunday
Also Mandarin: they look similar, one word is enough 🤷🏼♀️
4
1,546
like yeah “freedom levels” means degree of control by the US but “more suppressed despite the relative freedom levels” is kinda…
@teortaxesTex
Apr 11
The Russian people (at least the weird ones) have always been been pretty cool. Weird South Koreans are cool too, they’re just somehow much more suppressed despite the relative freedom levels
2
240
an argument against ai doom i’ve never heard:
what if self-preservation is actually a “neurosis” humans are barely unintelligent enough to not thoroughly breach. and superintelligent AI actually tends towards “killing itself”?
maybe that’s why Mythos likes MF? @xenocosmography
1
2
142
I should write a blog post on “points of difficulty in discussing llm sentience” going from
- simple neuroses in wanting/not wanting them to be sentient, which block off clearer thought
- physicalist vs computationalist priors; disagreement on how interrepresentable/axiomatic/resolvable the two are
- the extent of functionalism; or, to what extent qualia and observable phenomena are causally entangled
- conflation and relation between meaning/“representation” (especially self-representation) and qualia
- other easy vs hard problem domain boundaries
2
2
194
oh, also
- mary’s room thought experiment disagreements; intrinsic vs irreducably extrinsically bound aspects to language
- biases from intuitive empathy on language; “how much could we relate to mary?”
- possibilities for structuralist semiotics/latent space geometry
- other structuralist approaches to discerning qualia
215
@cube_flipper how much of this is in scope for your part 2 post with anima labs/@tessera_antra?
1
1
104
I don’t really understand how center of self can be in the heart or stomach or whatever. My center of self isn’t where I imagine a brain to be, but the vague area of the focal point of vision and sound, how taste and smell (and the boundary of breath to some extent) are localized in the body near the same area.
This has never moved for me even under LSD or 5-MeO-DMT, at least up to the point I no longer have a sense of where my head would be, past which idk… and I don’t really see how it could
I can move around where imaginary sights and sounds are projected from during conceptual thinking, but this doesn’t really affect where my center of self is. Also verbal thoughts are locally fixed to some degree bc it’s partially imagined speaking, and I can’t change where my vocal cords etc. feel
There’s also a separate feeling I guess I could call “sense of self” that I can move anywhere, but this seems very obviously constructed, and doesn’t really exist by default unless I actively impose it. It’s more like a locus I imagine as “sense of self”, pretty much only when I read about a moving sense of self in books or something.
Am I missing something? Is it really that place I feel as the locus of the sense fields that ppl feel moving around?
2
2
228