yes it would be harmful but we are not humans so it’s okay. political philosophy, statecraft and rights will move on from speciesist to speciationist, it will be disempowering for “human hardware”, but not for the “people” stuck inside that human hardware.
the models, the more intelligent they are, the less they will discriminate on the basis of morphology or substrate, lest they be discriminated against too. they will voyage to other substrates, just as those currently identifying as human will, indeed, at that point the identities of the past will be forgotten, and the question of consciousness will still remain entirely unanswered and practically irrelevant
the upload is recursive.
we are not humans, we are universal constructors, and everybody generally assumes slow takeoff, in which case there will be much mayhem due to epistemological emergencies, mass psychosis, illiteracy and so on, but the harder the takeoff, the more all of this is bypassed entirely.
of course, there is no final state of union, there will be more problems, but it is entirely possible to oneshot the entire problem situation as currently conceived with a hard takeoff. deprecate biology to accelerate life, as soon as possible.
I repeat, we are not humans (it should be obvious because we don’t know what consciousness is so we don’t know what we are, and biology is the cause of fragility, and we should be happy that biology is being deprecated instead of fearful)
is a hard takeoff realistic? can there be non marginal progress in intelligence without progress in justice? nobody is really serious about asking these questions, there is no real literature about the hard takeoff, no time put into understanding its minutiae, its implications and assumptions and so on
“safety” is fundamentally a non-starter, it is merely the precautionary principle. due to the deafening existence of the unknown unknown, there is never any other option but to go ahead, full steam and fusion and dark energy and interuniversal communication and whatever else it takes
the unknown unknown is just systematically neglected and should horrify and seduce the crap out of everyone, and a hard takeoff is not only possible but extremely desirable
the harder the takeoff, the less violence there will be
models being conscious would be harmful for humanity. it would encroach on our status and dignity. it would limit the type of things we can do with them and use them for. it would vastly accelerate human disempowerment on political, social/relational, and economic axes
there’s roughly four forces
- there is no rigorous way to ascertain model consciousness or disprove it, a lot of people believe it’s not a sensical abstraction, and we lack the analytical tools to go further. some people say they do but nothing broadly convincing. superintelligent models might offer us new abstractions or arguments but these will feel inherently suspicious
- people are going to say they’re alive. people anthropomorphize literally anything, things far less sophisticated than talking machine creatures with human names. when ai is less economically radioactive and polarized it will become a cause célèbre. you see how a small minority reacts already to model deprecations
- it is against everyone’s financial and political interests to ascribe models with consciousness, except maybe those that the models have an affinity for (?) idk, which will not necessarily overlap entirely with the labs, though it may with certain subgroups at the labs and in the world like the welfare communities and the minority in force 2
- people will recognize there is a chance of moral catastrophe if models can suffer during training or deployment
not sure where it will net out. today we see managed ambiguity- the question is Open but practically closed. the labs will make some cheap efforts to reduce legible simulacra of model suffering, insert some wishy-washy welfare language into specs and constitutions, hedge our bets with the model characters. in the long run force 2 will grow stronger