A multi-dimensional data container and computational engine optimized for big-data applications.
Joined October 2019
- Tweets 74
- Following 4
- Followers 60
- Likes 3
39 Photos and videos
Pinned Tweet
9 Jul 2024
#Caterva2 cannot only be used for sharing your compressed datasets in the internet, but also to efficiently perform operations on datasets exceeding available memory.
Look at our new blog explaining how this works 👉 ironarray.io/blog/caterva2-l…
Make compression better 😀
208
15 Jul 2024
Honored to put our team and knowledge in this work!
📢 Learn how using Blosc2 and Btune is improving the compression ratio of data (both lossless and lossy) coming from photon sciences. We were able also to reach compression speeds exceeding 23 GB/s.
Report 👉blosc.org/docs/LEAPS-INNOV_W…
Make compression better 😀
15
28 Oct 2022
ironArray joined Twitter 3 years ago already? Really? Proud to see how much progress we did during these intense years! 🎉🎉 🚀🚀
1
28 Oct 2022
3
2
13 Oct 2022
Reductions are the bread and butter of everyday data analysis. ironArray allows doing this for on-disk data faster and consuming much less memory (up to 20x) than e.g. Dask Zarr.
ironarray.io/docs/html/bench…
Do more with less.
1
1
7 Oct 2022
User Defined Funtions are a data mini-language (DSL), where code is intertwined with compressed data at a fundamental level to achieve best performance.
See how to create libraries of UDFs for operating with your data at ⚡️ speed: 👇
blog.ironarray.io/estimating…
Do more, with less.
1
18 Sep 2022
🎉🎉New blog on estimating 𝜋 by throwing darts and using ironArray:
blog.ironarray.io/estimating…
In this installment: high quality random number generators, User Defined Functions and reductions. See how ironArray can consume much less time and memory, compared with #NumPy/#Numba 👇
2
1
16 Sep 2022
ironArray comes with a nice assortment of blazing fast random generators.
Check our tutorial here: ironarray.io/docs/html/tutor…
API: ironarray.io/docs/html/refer…
But as there is more to life than speed, our tests also guarantee that distributions pass the Kolmogorov-Smirnov test 👇
2
15 Sep 2022
ironArray implements a matrix multiply that consumes (much) less time & memory than other parallel solutions. Even on Intel machines, ironArray can go at a speed that is just 2x slower than NumPy MKL.
Read more 👉 blog.ironarray.io/matrix-mul…
Save time & energy with ironArray ⚡️🌲
3
4
14 Sep 2022
Persistent storage has been gaining performance by leaps and bounds since solid state disks introduction. Check how out-of-core computation can lead to huge savings in memory compared with other solutions, while also performing decently fast: blog.ironarray.io/out-of-cor…
2
3
13 Sep 2022
Did you know that ironArray comes with AI-driven compression algorithms that lets it easily adapt to your preferences?
SPEED: favor speed
CRATIO: favor compression ratio
BALANCE: balance among the two above
No more time wasted deciding the codec to use! 👉blog.ironarray.io/surpassing…
2
3
12 Sep 2022
Did you know that our array constructors work in parallel and can go more than 10x faster than NumPy?
And that by leveraging compression you can host way more data using the same memory resources?
More info 👉ironarray.io/docs/html/bench…
Enjoy speed and compactness with ironArray!
2
1
12 Sep 2022
Did you know that our Enterprise license comes with up to 100 hours of support (negotiable)?
We want to *collaborate* with you to make your interaction with ironArray (both the library but *also* the company) as fluid and interactive as possible.
Your success is our fulfilment!
2
4
🎉We are happy to announce that we have moved most of our documentation into blosc.org/ (see the new navigation bar on top).
The doc pages in readthedocs.io have been removed for now. Sorry for the inconvenience.
Thanks to Marc Garcia, for all the help!
3
1
7 Sep 2022
Did you know that ironArray can access #Zarr remote arrays in a transparent way via the Zarr proxy array?
See our tutorial here: 👇 ironarray.io/docs/html/tutor…
2
10
11 Jul 2022
Often times you need to think about a problem from the inside-out. This is how we implemented User Defined Functions to re-define computing for a more cost effective usage of your computing facilities:
blog.ironarray.io/inside-out…
3
5
6 Jul 2022
Surpassing the memory wall requires a wise and well balanced combination of computation and compression, at the right time *and* at the right place.
Here is how we are achieving this:
medium.com/p/a4351cbc196
4
6
24 Jun 2022
In latest 2022.2 release we introduced *parallel constructors* to build arrays using multithreading.
This, combined with fine-tuned random generators, allows for amazing speed-ups and memory savings. Check it out in our latest tutorial:
ironarray.io/docs/html/tutor…
1
2
3
24 Jun 2022
And the random Poisson (integers) constructor is up to 40x faster, while consuming 10x less memory (with lossless compression):
1
1
2
24 Jun 2022
If you need constructors tailored to your needs, you can easily code them via our powerful User Defined Functions (UDFs).
E.g. the UDF below can build a triangular matrix at 3x the speed of NumPy and up to 5x faster than Numba (fully multi-threaded), while using 100x less memory.
1
2