With AI, it's easy to fall into the trap of overpromising and underdelivering.
There are so many AI products out there... but most of them are just shiny prototypes, not working products.
So, if your AI actually works, you have a chance to stand out. But how do you prove it? How do you demonstrate that your AI product is not like the others?
At
@CycleProduct, we came up with a concept we called the AI relevancy score. It's basically a calculation of our feedback AI's success rate broken down by workspace.
Every time a piece of feedback gets processed in Cycle, we create a score that compares the AI-generated answer to the "good" answer (ie the user-verified answer).
We use that score to 1/ understand general AI gaps in order to improve it for all customers, and 2/ spot specific workspaces for which AI performs poorly so we can further calibrate them.
It's extremely helpful for us as we can now iterate on our AI logic in an analytical way. But we're about to take it further 👀 We'll make that score available in the app.
That way, teams can track how Cycle's AI performs for their specific use case and compare it to alternatives. They can also see how calibration impacts relevancy over time, doing so building trust in the AI system.
If your AI is not bullshit, you gotta prove it. Be analytical about it. Show don't tell that you're the best AI out there. Go and compute that AI relevancy score!