Optimizing Data-to-Learning-to-Action, sequel to The Learning Layer. Data-to-Learning-to-Action® & Learning Layer® are ManyWorlds® brands!
Joined October 2009
- Tweets 977
- Following 437
- Followers 296
- Likes 364
172 Photos and videos
Optimizing Data-to-Learning-to-Action retweeted
17 Feb 2024
These debates all ultimately converge to Searle's Chinese Room Argument. As I discussed 13 years ago, what Searle's thought experiment leaves out is a capability for learning, which GPT-4 and Sora have. But Searle's meme still infects many. learninglayer.wordpress.com/…
I see some vocal objections: "Sora is not learning physics, it's just manipulating pixels in 2D".
I respectfully disagree with this reductionist view. It's similar to saying "GPT-4 doesn't learn coding, it's just sampling strings". Well, what transformers do is just manipulating a sequence of integers (token IDs). What neural networks do is just manipulating floating numbers. That's not the right argument.
Sora's soft physics simulation is an *emergent property* as you scale up text2video training massively.
- GPT-4 must learn some form of syntax, semantics, and data structures internally in order to generate executable Python code. GPT-4 does not store Python syntax trees explicitly.
- Very similarly, Sora must learn some *implicit* forms of text-to-3D, 3D transformations, ray-traced rendering, and physical rules in order to model the video pixels as accurately as possible. It has to learn concepts of a game engine to satisfy the objective.
- If we don't consider interactions, UE5 is a (very sophisticated) process that generates video pixels. Sora is also a process that generates video pixels, but based on end-to-end transformers. They are on the same level of abstraction.
- The difference is that UE5 is hand-crafted and precise, but Sora is purely learned through data and "intuitive".
Will Sora replace game engine devs? Absolutely not. Its emergent physics understanding is fragile and far from perfect. It still heavily hallucinates things that are incompatible with our physical common sense. It does not yet have a good grasp of object interactions - see the uncanny mistake in the video below.
Sora is the GPT-3 moment. Back in 2020, GPT-3 was a pretty bad model that required heavy prompt engineering and babysitting. But it was the first compelling demonstration of in-context learning as an emergent property.
Don't fixate on the imperfections of GPT-3. Think about extrapolations to GPT-4 in the near future.
1
1
99
Shane Legg, cofounder of DeepMind, explains the importance of adding Search to Neural Networks.
Background: "Move 37" was a pivotal play by AlphaGo during the 2nd of its 5-game series against Go Champion Lee Sedol. AlphaGo placed a stone in a highly unconventional position that even confused experts, who first deemed it an error. Yet as the game unfolded, the move proved to be strategically brilliant, opening up opportunities that only became clear much later.
In a sense, Search increases intelligence at inference. The AI can take more time to deliberate without altering its parameters, and dynamically tradeoff efficiency with deeper thinking.
Video credit: @dwarkesh_sp podcast w/ @ShaneLegg
39
180
978
242,201
Optimizing Data-to-Learning-to-Action retweeted
11 Jul 2023
AI is transforming science, which provides the foundation for much of overall innovation in the economy, thereby representing an additional driving force on innovation most generally (already being accelerated by AI). technologyreview.com/2023/07… Example: science.org/content/article/…
1
1
36
“With bigger models, you get better performance, but we don’t have evidence to suggest that the whole is greater than the sum of its parts.” hai.stanford.edu/news/ais-os…
10
Optimizing Data-to-Learning-to-Action retweeted
21 Mar 2023
Delighted that the two startups for which I provide technology and IP guidance are partnering to deliver the next-generation #machinelearning-driven media processes to media clients right now.
21 Mar 2023
revealit TV and I are proud to announce our partnership with DCO.ai to use the revealit TV platform to elevate viewer experiences and monetise some of the largest media catalogs in the world.
blog.revealit.tv/2023/03/21/…
1
2
2
110
Optimizing Data-to-Learning-to-Action retweeted
21 Mar 2023
AI enthusiasts have cried wolf for decades.
GPT-4 is the wolf. I've seen it with my own eyes.
betonit.substack.com/p/gpt-r…
20
78
649
400,180
Optimizing Data-to-Learning-to-Action retweeted
6 Jan 2023
It's amusing how the homework threat is always top of mind when it comes to #gptchat, etc. Rather than the threat to the relevancy of what is being taught and graded.
5 Jan 2023
Thx to Scott Aaronson, GPT outputs will soon be watermarked w/ a random seed, making it much harder to submit your GPT-written homework without getting caught
He doesn’t give too many details about how it works, but I suspect its possible to bypass using a clever decoding strat
2
1
63
Artificial innovation: "human experts 'favored' some amino acids over others, sometimes leading them to incorrect choices. Also, the computer program correctly pointed to some proteins with qualities that didn’t make them obvious choices for self-assembly" rutgers.edu/news/latest-huma…
2
2
Optimizing Data-to-Learning-to-Action retweeted
14 Oct 2022
Human-->Machine-->Human innovation synergy arstechnica.com/information-…
4
4
"bring imagination to life and create one-of-a-kind videos full of vivid colors, characters, and landscapes. The system can also create videos from images or take existing videos and create new ones that are similar." ai.facebook.com/blog/generat…
1
"researchers have shown that the hippocampus, a structure of the brain critical to memory, is basically a special kind of neural net, known as a transformer, in disguise" quantamagazine.org/how-ai-tr…
The robots have come:“We’re watching the death of artistry unfold before our eyes”vice.com/en/article/bvmvqm/a…
Optimizing Data-to-Learning-to-Action retweeted
14 Aug 2022
In other words, identify and resolve the limiting constraint. This is universal, yet is too often unrecognized (which leads to misinvestment) and applies to all learning or innovation processes. datatolearningtoaction.com/
4
11
Optimizing Data-to-Learning-to-Action retweeted
14 Aug 2022
An idea from the history of science to generate startup ideas or ways to help in a crisis: find the reverse salient.
A reverse salient is the technology or process that is holding back development of the whole system (like ⚡️car batteries did). Solving the salient unlocks change
6
35
266
Optimizing Data-to-Learning-to-Action retweeted
5 Jul 2022
Just how much have language models grown in the last 4 years? Let's have a look. In 2018, the puny BERT “large” model premiered with a measly 354M parameters. It can be trained on a single 8xA100 node in 5 days. That costs $2K on AWS - almost free by LLM standards! 🧵
12
233
1,240
Optimizing Data-to-Learning-to-Action retweeted
24 Jun 2022
Humans ascribe sentience to ELIZA but not cows. So when people say "humans over-ascribe sentience", they are correct - our rate of false positives is high. And when people say "humans under-ascribe sentience", they are also correct - our rate of false negatives is also high.
3
6
95
Optimizing Data-to-Learning-to-Action retweeted
16 Jun 2022
The Fundamental Equation of Economics (and life).
For any project, activity, or most broadly, action:
Total Value = Direct Value Learning Value
researchgate.net/publication…
2
2
Be decision-driven not data-driven! Business value is ultimately driven by data-to-learning-to-action processes, and the way to optimize them is by working backward from the decision, not working forward from the data. towardsdatascience.com/the-d…
1
2
4
Optimizing Data-to-Learning-to-Action retweeted
23 May 2022
By the time a human child turns 5, she has seen the equivalent of 800 million "frames" of video audio touch to learn how the world works through Self-Supervised Learning.
Much of it is acquired actively.
5 years * 365 days * 12 hours * 3600 seconds * 10 fps = 788.4 million
144
397
2,796
Optimizing Data-to-Learning-to-Action retweeted
13 May 2022
In Feb-Mar 2020 most people underestimated COVID because the human brain doesn't intuitively register exponential growth. Are people likewise generally underestimating the time to #AGI?
12 May 2022
Gato🐈a scalable generalist agent that uses a single transformer with exactly the same weights to play Atari, follow text instructions, caption images, chat with people, control a real robot arm, and more: dpmd.ai/Gato
Paper: dpmd.ai/Gato-paper 1/
2
2