Amazing panel on impact measurement with
@bbeats1 ,
@0xyNaMu &
@feemschats
Some key points i remember off the top of my head
- ideally we shouldn't have evaluation as the mechanism is self regulating
An example is
@ProtocolGuild which adds ethereum core devs to their list for automated funding. With every extra person added the remaining members get a smaller proportion of the pie , which is balanced by loss of legitimacy in not adding eligible members
- impact is always relative to cost, for example educating ten kids but for a million dollars is negative impact.
The way to calculate impact is quantify the outcome , subtract the counterfactual of the outcomes that would have occured even if the intervention didn't happen (not all 10 kids graduated because of you, some would have anyways), and then divide quantified benefit by cost incurred
- most significant change technique is a good methodology for impact evaluation in web3 projects, document case studies of significant change because of you & then reshare with grantees
- we need to move beyond popularity contests for allocation mechanisms, which both
@gitcoin QF and
@Optimism retro pgf currently are.
Multiple evaluators each quantifying a grants benefit cost ratio with help of LLMs & then distributing funding to from highest mean or median BC ratio to lowest is one way of doing so
- try to negotiate deals that aren't just free money to projects
Token swaps as investment & being a customer of products already built are better ways of ecosystem support that leverage the power law and support projects already killing it
- evaluation metrics should either be binary (did product get built or not) or quantitative (how many users did it get)
- we need to get more web2 evaluators in our space so we don't repeat the mistakes already made
Amazing discussion on the ins and outs of impact measurement. Where things stand in the web3 space right now as well, where we hope to get to one day as well as the tools to use 🤝