Evidently has 5000 ⭐️ on GitHub 🎉🎉🎉
Thank you for using Evidently and helping us make it a trusted open-source ML and LLM observability framework!
Let AI observability be available to all!
To support the project, give us a ⭐️:
github.com/evidentlyai/evide…
🚀 New release: Evaluate LLMs with Evidently open-source!
Reports, Test Suites, and a Monitoring dashboard with lots of methods:
⚖️ LLM-as-a-judge
🤗 ML-based scoring
📐 Semantic similarity
💬 Text stats/patterns
Tutorial: docs.evidentlyai.com/get-sta…
GitHub: github.com/evidentlyai/evide…
And how did you spend your Saturday? 👀
A spoiler preview of mine👇🏽
Working on the new ML Observability Course for @EvidentlyAI!
Register here: evidentlyai.com/ml-observabi…
Our CTO, @EmeliDral, will be speaking at the AI Superstream on Sept 13!
Stop by to learn how to evaluate quality and drift in text and multimodal data 🔡
Thanks @OReillyMedia for inviting us and @fab_clemente for hosting the session!
Join for free 👉 oreil.ly/NLdJ-
⏰ Staying Alert: how to continuously test ML models in prod.
It was really fun to speak at @PyConDE. They now published the recording of the talks!
🚦 How to automate ML test thresholding?
💻 How to compute complex metrics?
🏗 How should you retrain?
youtube.com/watch?v=4BgmBtsS…
🛠 How do top engineering teams apply ML to improve products and processes?
We put together a list of 200 case studies that share practical ML use cases and learnings from designing ML systems: evidentlyai.com/ml-system-de…
Here are our five favorites 👇
1/7 🧵
📊Meet Evidently 0.3.2.1!
Evidently HTML Reports finally got more lightweight!
Great news for anyone working with larger datasets!
By default, we will show aggregated plots. If you want the old look, choose a "raw data" render option.
Release: github.com/evidentlyai/evide…
🚀Meet @EvidentlyAI 0.3.1.
💬 Text descriptors are supported in all relevant metrics! You can run tests like "check that text length is within these bounds."
🔢 Embeddings drift detection in Tests Suites
🐞 A lot of fixes (thanks to everyone reporting!)
github.com/evidentlyai/evide…
New release! @EvidentlyAI 0.3 is here!
🆕 Embeddings drift detection with 4 methods
✅ Customize JSON output to define what to include
📚 Use text descriptors in tabular checks (e.g, “test that text length is between X and Y”)
⛔️ Old API removed
Release:
github.com/evidentlyai/evide…
🤔 Why we do ML model retraining wrong, and how to do better
Our CTO, @EmeliDral, spoke on @PyDataGlobal 2022.
She discussed how to design a pragmatic ML model retraining strategy, considering available resources, actions and costs of error👇
youtube.com/watch?v=1LcWdV6D…
📚How can you evaluate data drift on text data?
We’ve just released the latest functionality in the @EvidentlyAI open-source library that does just that.
Here are the two approaches that we implemented. 👇🏼
(1/4)
Text descriptors drift.
You can also look at the shift in the text properties - such as length, out-of-vocabulary words, or non-letter symbols.
You can then compare the distribution of the text descriptors between the two datasets.
(3/4)
The latest release contains a lot more!
📊A new Text Overview preset
📚All relevant Evidently tests and metrics support texts.
🔡Support for multi-model datasets.
Check it out!
GitHub: github.com/evidentlyai/evide…
Release blog: evidentlyai.com/blog/evident…
Meet @Evidently 0.2, a stable release of our open-source ML monitoring tool:
👩🏽🔧 A major refactoring
🧱 100 ML tests and metrics
🚦 Simplified API
🛠 Integrations with Airflow and Mlflow
Getting Started tutorial 👇
Video: youtu.be/2d5AgPKbthE