we have been thinking recently about the need for something like "AI privilege"; this really accelerates the need to have the conversation.
imo talking to an AI should be like talking to a lawyer or a doctor.
i hope society will figure this out soon.