Bug fixes & analysis for Qwen 2.5: 1. Pad_token should NOT be <|endoftext|> Inf gens 2. Base <|im_start|> <|im_end|> are untrained 3. PCA on embeddings has a BPE hierarchy 4. YaRN 128K extended context from 32B 5. Fixed versions 128K GGUFs: huggingface.co/unsloth Details: 1. Pad token bug - for finetuning, never use pad_token = EOS - this will result in infinite generations since finetuning will ignore them. Base model also has a chat template - remove this. @UnslothAI versions fixed them 2. Untrained tokens issues. Do NOT use the Qwen 2.5 chat template for the base version - <|im_start|>, <|im_end|> are untrained since Norm(<im_x>, pad_token) is close to 0. Instruct version have them trained. 3. PCA on embeddings for Base and Instruct show a BPE hierarchy. Less frequent tokens are obvious since they're ordered by ID. PCA shows <|im_x|> moving away from being untrained. Same phenomenon for Llama & more models. 4. Uploaded native 128K extended YaRN GGUFs for Coder 0.5B all the way until 32B to huggingface.co/collections/u…. Use the 128K version for long contexts. Use the 32B native version for general chats. Also, Unsloth can finetune 14B in a free Colab! Conversational style finetuning: colab.research.google.com/dr… Kaggle 14B notebook: kaggle.com/code/danielhanche… Unsloth can also finetune the 72B variants in a 48GB card!

thinking about what "libraries" might look like in a future where agents do most coding: first, coding and taking action probably won't have a hard line between them. we already see the start of this with code interpreter where the code is just some ephemeral thing. pre llms, these are separate because humans write code slowly; we work on the order of multiple minutes for even the simplest of code, but agents will write at hundreds, maybe thousands of tokens/s. for agents coding will be like moving a muscle okay that was kind of a side point. more interestingly, what happens to libraries? currently big libraries serve as schelling points for functionality. if i need to do some data analysis thing, i don't have to search through all code that's ever been published, i can trust that it's probably in pandas. this is better because my search bandwidth is limited: i only have so many apm, and can only parse so many tokens per second. however, we're good at semantic chunking, so we package a bunch of related functionality into one shortcut chunk that we integrate into our knowledge--a library that's not a limitation for agents though. or rather, to an agent it probably doesn't matter as much if there's a shortcut to the search via a big library, versus a flat index of a bunch of granular functionality. so for example, imagine if we had a "component index" where instead of indexing at the level of a library, we index at the level of a function or data structure. we still have dependencies, and each individual function or structure is arbitrarily complex. i can imagine that some future agent empowered with search can find any relevant functionality they need this way in seconds is this "better" for agents? i think there are advantages. dependency management would be way easier--a component would only have the exact dependencies it specifically needs, rather than the dependencies for the whole library it's packaged with. you can get the stuff you need a la carte. migration to new versions can be done for an individual component rather than all components that are part of a library i'm not sure if this will happen though. libraries are certainly a local optimum currently, and the simple fact that the big libraries are well represented in training data means that they're "good enough" but suppose we progress far enough along where the management of libraries are done by agents. i can see a world where at some point work on existing libraries gradually wind down and they become effectively frozen, and only minor non-breaking changes are allowed on them. and around them, an array of single-component libraries pop up, built on top of these artifacts. maybe over time specific components within the old libraries are reimplemented outside of the library as standalone components