Paper accepted to NAACL 2024 main conference. arxiv: arxiv.org/abs/2404.02421 In this work, we explore the interaction between two bottlenecks of LLMs: negation and tokenization (quoting @karpathy: "tokenization is the root of suffering").

10/ We see that models could clearly understand negation despite incorrect tokenization. This could be an interesting phenomenon to look at when discovering LLM interpretability. Also, as English is poor in morphology, we are eager to extend this analysis to other languages.