ML perf is not only about writing faster kernels but also about speeding up disk to GPU transfer and improving setup times. i love how FlashPack (made by fal) aims to solve the former. should i write an article going deeper into how it works?

I fuckin love flashpack Friday🔥🔥 Awesome onion my guy😎

We released Diffusers 0.38.0, and it's packed with new pipelines and several library-related improvements 🔥 A bunch of new pipelines, including audio 🎼 * Ace-Step 1.5 * LongCat-AudioDiT * Ernie-Image And more! Next up, we added support for: * Flash Attention 4 * Loading with FlashPack * Ring Anything as a new backend for context parallelism Last but not least, we added an example on how to profile a DiffusionPipeline and potentially improve its performance. Enjoy 🧨

If you transferred $PACK from @Kraken to @hashpack, you got a KrakPACK 🐙 If you don’t have any $PACK in your HashPack, you got a LackPACK 😢 If you got @hBUDS_ in your HashPack, you got a StashPACK 🍃 $PACK If you got @HMNKYs branch swinging inside your HashPack, you got a BranchPACK 🐒 $PACK If you got @quackinals quacking 🦆 in your HashPack, you got a QuackPACK $PACK If you got the @McLarenF1 racing collectibles in your HashPack, you got a FastPACK 🏎️ $PACK If you got @builtbyslime SLIMES 🟢 in your HashPack, you got a SlimePACK $PACK If you got 1000 NFTs in your HashPack, you got a GrandPACK $PACK If you got a bunch of throwback NFT’s in your HashPack, you got a ThrowPACK $PACK If you get flashbacks looking at your HashPack, you got a FlashPACK $PACK If you haven’t cleared the cache on your HashPack, you got a CachePACK $PACK If you burned through all the $HBAR in your HashPack, you got an AshPACK 🔥 $PACK If you crashed out and insta-sold everything in your HashPack, you got a CrashPACK 💥 $PACK If you receive a steady drip of tokens into your HashPack, you got SplashPACK 💦 $PACK If you have multiple HashPacks, you can Compare & ContrastPACKs $PACK Let’s keep it goin 😎

was working on this side project for local media generation finally i can see some cat images😭 curr. its a mess, f16 on sdxl-turbo. although the load times are nice. claude helps a lot coz i hate writing gui i have about 100 features from flashpack to lora models support todos😭

Using the flashpack weapon ....grrrr

Diffusers形式ですね fal/Z-Image-Turbo-Control-2.0-FlashPack huggingface.co/fal/Z-Image-T…

昨日、恵比寿リキッドルーム豆柴の大群 6周年記念ライブりステップ 5人のイカサマダンスみれたり、ナオちゃんのソロ曲FLASHPACK見れて嬉しかったよ。最高の時間ありがとう。 #豆柴の大群

@flashpack | @Uber | @HBXGroup_ | @ScapadeApp | @roomex | @SimplyOwners | @Price_Labs | @PasstheKeysUK | @eviivo | @SAPConcur #TravelTech #TechTrends #Hospitality #AIInTravel

Introducing FlashPack: Lightning-Fast Model Loading for PyTorch The FlashPack package dramatically speeds up PyTorch model loading by flattening all weights into a single contiguous stream, memory-mapping the file, and overlapping disk, CPU, and GPU operations with CUDA streams. This approach yields 3-6× faster loading compared to traditional methods like load_state_dict(), reducing GPU idle time and improving overall performance, especially on syste... blog.fal.ai/introducing-flas…

HDMI flashpack 2020

A lot of confusion over FlashPack. The key innovation that I think is being overlooked is it doesn’t use load_state_dict.

This is huuuuge 🤯 Try FlashPack now!

Really happy we were able to open-source this. Internally, we have been using early versions of FlashPack to speed up model loading, but @MLPBenjamin improved the loading substantially, and packaged it up wonderfully tldr; 2.5x vs safetensors, here’s why pytorch’s standard `load_state_dict` leads to many tiny CUDA allocations and copies. However, if you flatten the weights to a single tensor, you can do a single device allocation, and use a mmapped buffer to efficiently copy the weights to the device memory. However, the model params expect to have separate tensors, which make efficiently by using views.

fal open-sourced flashpack: it takes way less time to load the models to the GPUs, and works just as fast in multi-gpu environments as well.

Try FlashPack here! github.com/fal-ai/flashpack

🚨 Introducing FlashPack: Lightning-fast model loading package for PyTorch! ⚡ 3-6x faster model loading than current methods 📦 Convert existing checkpoints in one command 🔧 Works on any system Read our blogpost for more details!👇️ blog.fal.ai/introducing-flas…

Arrive solo, leave inspired. Built the brand for FlashPack.

Awesome!!!

Another successful youth football camp in the books. 20th consecutive camp. Thank you to all the players - past, present and future flashes. #GoBlue #FLASHpack