Bookmarking wikenigma.org.uk/start

I loved gevent. It was bringing all the benefits of event loop I needed and leaving me with a straightforward API on monkey patched threads. I never could understand why it was treated as an ugly child

We all love short papers fermatslibrary.com/s/the-uns…

Nice, I might be eventually able to use letter “m” in passwords for some, ancient services. But then again if they are already ancient, will their CISO actually care about the new NIST guidance? mastodon.social/@LukaszOlejn…

Explains why I found myself forced to not just block Musk, but also mute the terms “Elon”, “Musk”, “Elonmusk” to get a Twitter experience where I wouldn’t have every second tweet of his on my timeline. Case study worthy on how you degrade a social network long-term

Ruby on Jupyter nbviewer.org/github/SciRuby/…

Swift.org - Announcing Swift Homomorphic Encryption swift.org/blog/announcing-sw…

As Apple Intelligence is rolling out to our beta users today, we are proud to present a technical report on our Foundation Language Models that power these features on devices and cloud: machinelearning.apple.com/re…. 🧵

White-Collar Work Is Just Meetings Now - The Atlantic theatlantic.com/ideas/archiv…

Long awaited - standardised way to run web app in lambda github.com/awslabs/aws-lambd…

We live in the future; it's just not distributed evenly: hakaimagazine.com/videos-vis…

Intellectual obesity - I love the term: gurwinder.blog/p/the-intelle…

Next time you estimate development think about leaky pipelines: hiandrewquinn.github.io/til-…

Convolutional Neural Networks in action

Yann LeCun says he is working to develop an entirely new generation of AI systems that he hopes will power machines with human-level intelligence. It could take up to 10 years to achieve, he tells the @FT in an interview on.ft.com/3KbShLF

I'm playing with G-Eval to test the LLM outputs using LLM. It roughly works until it doesn't. How am I supposed to reason with test result: "the actual output's prompt is in Polish which mismatches the language-prompt specified as Polish, aligning correctly" #llm #gpt #deepeval

honestly, Word, we have enough CPU to keep the table of contents updating automatically

I’ve just been just told by the staff at Pret A Manger that there is no water in espresso 🙈

How to be as "smart" as Auto-Regressive LLMs: - memorize lots of problem statements together with recipes on how to solve them. - to solve a new problem, retrieve the recipe whose problem statement superficially matches the new problem. - apply the recipe blindly and declare victory. - do not use basic logic. - do not use common sense to check your solution. - do not use a mental model of the situation as a sanity check. - do not simulate the scenario in your mind using your world model. - when someone tells you your solution is wrong, reply "I'm sorry, you are right" and apply another irrelevant recipe. Knowledge accumulation is not a substitute for actual understanding.

At this point I feel like we understand pretty well what's going on with LLMs: - Outputs are roughly equivalent to kernel smoothing over positional embeddings (arxiv.org/pdf/1908.11775.pdf) - The learned computation model is *probably* bounded by RASP-L (arxiv.org/pdf/2310.16028.pdf) - LLMs learn structure primarily from human generated content (text, images) which is far more structured and predictable than the universe. - LLaMa3 shows us that the higher quality the annotations on the human generated content, the better the LLMs do (10million messages is a lot!) - Multi-turn labeling is currently very expensive and so likely driving the costs of the models. - Right now we're likely bottlenecked not on CPU, or size of data, but number and quality of annotations. So tl;dr. Great at predicting what a human would do or say by averaging in distribution data in the corpus. No emergent generality. Currently bottlenecked by high quality annotated data. @hausdorff_space did I miss anything?