Large Language Models don't reason. Thank you, Apple.

Nice Paper for a long weekend read - "A Primer on the Inner Workings of Transformer-based Language Models" 📌 Provides a concise intro focusing on the generative decoder-only architecture. 📌 Introduces the Transformer layer components, including the attention block (QK and OV circuits) and feedforward network block, and explains the residual stream perspective. It then categorizes LM interpretability approaches into two dimensions: localizing inputs or model components responsible for a prediction (behavior localization) and decoding information stored in learned representations to understand its usage across network components (information decoding). 📌 For behavior localization, the paper covers input attribution methods (gradient-based, perturbation-based, context mixing) and model component importance techniques (logit attribution, causal interventions, circuits analysis). Causal interventions involve patching activations during the forward pass to estimate component influence, while circuits analysis aims to reverse-engineer neural networks into human-understandable algorithms by uncovering subsets of model components interacting together to solve a task. 📌 Information decoding methods aim to understand what features are represented in the network. Probing trains supervised models to predict input properties from representations, while the linear representation hypothesis states that features are encoded as linear subspaces. Sparse autoencoders (SAEs) can disentangle superimposed features by learning overcomplete feature bases. Decoding in vocabulary space involves projecting intermediate representations and model weights using the unembedding matrix. 📌 Then summarizes discovered inner behaviors in Transformers, including interpretable attention patterns (positional, subword joiner, syntactic heads) and circuits (copying, induction, copy suppression, successor heads), neuron input/output behaviors (concept-specific, language-specific neurons), and the high-level structure mirroring sensory/motor neurons. Emergent multi-component behaviors are exemplified by the IOI task circuit in GPT2-Small. Insights on factuality and hallucinations highlight the competition between grounded and memorized recall mechanisms.

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time. It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

ChatGPT can now create Mind Maps. No more wasting hundreds of hours making visuals for studying or simplifying complex ideas. Here’s how to do it for free in a few seconds:

Chatting with @GroqInc’s CEO @JonathanRoss321. Groq has super fast token generation capabilities now. And, I was excited also to hear about his plans to scale up capacity aggressively and also expand this to other models than just LLMs! This is a good time to be building AI applications.

OpenAI'a Sora is the best example of Synthetic data example. Hard to replicate such a moat in an enterprise but if we can get the right distribution of the data and its attributes, I think we can see better models for the basic use cases of the enterprise. cc @DevendraDesale

Finally free from stranglehold of Facebook products. Moved over 40 friends to signal over last weekend and finally successful in #Deletewhatsapp.

Excited to be networking from home on @lunchclubai! Use my invite link to skip the waitlist and meet interesting people over video: lunchclub.com/?invite_code=d…

The fastest route is not always a straight line. x.com/knowIedgehub/status/13…

People can be: 1) How-first 2) What-first 3) Why-first How-first people execute well. What-first people create well. Why-first people lead well.

Guy on the right is a Growth PM

Drones, data analytics, smart seeds: How to reforest x1,000 faster after wildfires bit.ly/36aEiiJ #bigdata, #datascience #ds

If you can treat genuine cases of defence officers this way I dont know how you treat the rest of your customers. Neither airline helped him and now he has to spend a hefty amount on next tiket and spend the night at airport.

scheduled departure. He is on defence duty and requested multiple staff members to accommodate him on the indigo flight but they flatly refused. Spice jet too did not accommodate him on the next flight to Vizag and did not even reimburse the missed flight.

Pathetic customer service by @flyspicejet and @IndiGo6E. First the Pune to Hyderabad #spicejet flight gets delayed by 3 hrs. Then after informing #indigo customer service of the late arrival , they refuse to let my brother board the flight even though he reached an hour before

Survival Analysis for Business #Analytics buff.ly/2AAiBNn

The AI Index is out! Page after page of interesting charts showing AI trends. aiindex.org/ For example, since 2013, the share of US jobs requiring AI skills has grown 4.5x! @yshoham

Top 10 #MachineLearning Videos on #YouTube, updated buff.ly/2p6HjeL