The Road Not Taken
Two roads — hard negatives and distillation — diverged in a yellow wood...
We looked down the hard negatives one as far as we could...
Then took the other — the distillation, as just as fair,
And empirically now shown to be the better claim…
🚨 New Pre-Print! You've just added your 600th model to your negative mining pool and filtered all false negatives. Does any of this even matter when we can apply distillation?
In this work with
@debforit and
@macavaney, we explore data selection in modern ranking. 🧵 Below