NEURAL NETWORKS FOR EMBEDDED DEVICES @Tesla's US12346816B2 patent introduces a transformative neural network architecture addressing the fundamental computational constraints of embedded devices through systematic bit-width reduction and arithmetic overflow prevention. The invention confronts the critical limitation wherein "processors may be too complex or expensive for use in inexpensive devices, such as IOT devices that may include inexpensive processors having a more limited bit-length" ([0002]), establishing a new paradigm for deploying sophisticated neural networks on resource-constrained hardware. This architectural innovation enables neural network inference on processors traditionally limited to simpler computational tasks, specifically 8-bit arithmetic processors found in IoT devices. The technical breakthrough manifests through a co-designed approach linking neural network topology with arithmetic constraints, wherein "dimensionalities determined such that an output value generated by combining elements of an input layer as maximum values of the first integer representation with elements of a corresponding filter as maximum values of the second integer representation does not overflow the bit length of the registers" (Claim 1). This mathematical guarantee ensures operational integrity within 8-bit non-saturating arithmetic environments while maintaining inference accuracy through strategic quantization and novel convolutional topologies. The patent's significance extends beyond incremental optimization, fundamentally reconceptualizing how neural networks interact with hardware limitations. Rather than treating bit-width constraints as performance degradation factors, the invention integrates these boundaries as first-order design parameters, yielding architectures that achieve optimal efficiency precisely because of, not despite, their computational constraints. [FIG. 1: Star-shaped convolution filter showing 5-element spatial sampling pattern with center and cardinal direction weights]

A tweet/post is how I initiated the dozen or so startups that constitute Tesla. Tesla has made very few acquisitions. Our growth is ~90% organic. We didn’t “play to our core strengths”, we created core strengths from nothing.

あり得ない快挙。 たった1.5Bのモデルが、強化学習で、4万問の数学の問題でトレーニング。たった60万円で数学能力、o1-previewを超える結果に。 これが真実で本当に再現できればま、特化型で、1.5B、企業向けの開発でボコボコ使われることほぼ間違いない成果。 やべーと言わせてほしい。 本文翻訳 カリフォルニア大学バークレー校は、15億の小さなモデルがRLによる数学でo1-previewを上回ることを実証しました。 彼らは、40K の数学の問題に対して Deepseek-R1-Distilled-Qwen-1.5B にシンプルな RL を適用し、8K のコンテキストでトレーニングした後、16K と 24K にスケーリングしました。 数学で O1 プレビューに勝つには、3,800 A100 時間 (4,500 ドル) が必要です。 最も優れている点は、モデル、トレーニング コード (ByteDance verl ライブラリに基づく)、データセットなど、すべてをオープンソース化したことです。

DeepScaleR-1.5B-Preview a open-source, 1.5B-parameter model trained with RL to surpass o1-preview for general math reasoning