@DIR_Workshop profile picture
DIR Workshop @DIR_Workshop
Joined September 2020
  • Tweets 175
  • Following 175
  • Followers 154
104 Photos and videos
Concluding remarks of our last session of DIR 2021 1/2. Future of IR metrics? #DIR2021 @gkazai
Meta question: How to access a metric? What would be a good metric for a metric? #DIR2021 @gkazai
1
Offline eval looks easy? Looks simple, what could go wrong? Think again #DIR2021 @gkazai @arjenpdevries @maria_heuss
2
How are the offline eval pipelines used? Ranking experiments are the customers. Ranking experiment's success is judged by the eval pipeline #DIR2021 @gkazai @arjenpdevries @maria_heuss
A system's perspective on IR evaluation #DIR2021 @gkazai @arjenpdevries @maria_heuss
How does evaluation look like with a real search engine? An industry eval landscape from Bing #DIR2021 @gkazai @arjenpdevries @maria_heuss
2
How to setup an offline IR eval pipeline? A guideline #DIR2021 @gkazai @arjenpdevries @maria_heuss
Looking for references for Offline IR evaluation? Check out these books #DIR2021
Good overview of the Metric landscape, from a practical/industry perspective. How to define it offline v/s online #DIR2021 @gkazai @arjenpdevries @maria_heuss
1
Online/user-related metrics are harder to gather. Logs are noisy #DIR2021
Ex: of a 'Metric' and it's 'proxy'. User retention v/s Relevance #DIR2021 @gkazai @arjenpdevries @maria_heuss
1
How do you 'operationalize' a metric? Ans: Use a proxy. #DIR2021 @gkazai @arjenpdevries @maria_heuss
What is a 'metric'? #DIR2021 @gkazai @arjenpdevries @maria_heuss
1
How is an IR system evaluated @gkazai #DIR2021 @arjenpdevries
History of IR evaluation @gkazai #DIR2021
DIR 2021 1/2 keynote kickstart by @arjenpdevries & @maria_heuss #DIR2021 (1/n)
1
📢 Closing Week DIR 2021 1/2 - One day to go! 🚨 ⚡️One day to go for the closing session of DIR 2021 1/2. Don't miss it!! 🤝Join us to learn about: Industry perspective on IR evaluation by @gkazai from Microsoft. 🌎 Register here: meetup.com/SEA-Search-Engine… @maliannejadi @mdr

DIR Workshop@DIR_Workshop
21 Feb 2022
🚨📢 Last week - Keynote Announcement 🚨📢 For week 4 of DIR 2021 1/2, we have a very interesting keynote talk by Gabriella Kazai @gkazai from Microsoft. Topic of the talk: IR Evaluation - An Industry Perspective. 🌎Register here meetup.com/SEA-Search-Engine… @maliannejadi @mdr
1
1
🚨📢 Last week - Keynote Announcement 🚨📢 For week 4 of DIR 2021 1/2, we have a very interesting keynote talk by Gabriella Kazai @gkazai from Microsoft. Topic of the talk: IR Evaluation - An Industry Perspective. 🌎Register here meetup.com/SEA-Search-Engine… @maliannejadi @mdr
7
11
This concludes week 3 of DIR 2021 1/2. This was the last week of lightning talks. ⚡️ Hope you all enjoyed it. Next week is the concluding week for DIR 2021, stay tuned for the latest updates. 😇
1
Want to try out the method and extend it? Link to code, and conclusions @HarrieOos #DIR2021
Load more