working with large datasets is a breeze with polars parquet. it's surprisingly scalable, worked fine for me at a scale of billions of rows and around 1tb of data. for charts the data needs to be preprocessed anyway, i try to keep it below 10k rows. for very dense charts (think 1m rows) use datashader, i posted about this on my blog.