⚖️ Workflow and incentives behind evaluating reports + negative impact

[marsrobertson]

How does the evaluation work?

Docs: https://testnet.hypercerts.org/docs/whitepaper/evaluation

My previous research

I was working on a highly aligned project called BaseX and applied to Kleros incubator:

The reason why I mention https://kleros.io is the community of the jurors - they are impartial experts when comes to fact-checking data. Already established community with expertise - a highly valuable asset.

Jurors can join the court where they have qualifications, for example, they know the language and can evaluate if the translation was done properly. For Hypercerts, a new category should be created: "Impact Evaluation"

Proposed workflow of evaluation

Noone knows the project better than the founders, participants, people on the ground. They have a much better understanding of the impact they are creating. It should be the Hypercert creator who can use standardized values - https://impactgenome.org/impact-index/ - they moved it away - need to use Web Archive - http://web.archive.org/web/20230307121254/https://impactgenome.org/impact-index/

Then the jurors will have a simple job:
✅ Yes, correct, the overall impact assessment is reasonable*
⛔️ No, wrong, there are some issues

Additional rules:

• *reasonable because evaluating impact is not the exact science, as long as the assessment is reasonable, it should be accepted
  • What is the definition of “reasonable”? Fair, just, accurate, sensible… There are loads of uncertainties in the climate and finding a perfect estimate is a fallacy. “Reasonable” is good enough. When in doubt ask AI.
• If the evaluation is rejected, we can differentiate honest mistake VS outright fraud
• Both projects and jurors can earn a reputation through soulbound tokens
• When new information becomes available - a new supplementary evaluation can be added TBD TBC
• Evolving, anti-fragile system, responding to any curve balls, bad actors will not outsmart us

Negative impact

Someone needs to keep an eye on the polluters. Who is watching the watchers, who is regulating the regulators? Entrepreneurs are much faster to respond to challenges than government.

Hypercerts can be applicable to both POSITIVE and NEGATIVE impact.

• For positive activities: minting Positive Value Token (PVT)
• For negative activities: minting Negative Value Token (NVT)
• PVT and NVT = names TBD TBC for something more eco-friendly and meme-able
• Accounting for PVT and NVT is done separately, rather than a single value, this is to preserve the fact that someone did damage in the past (and then it is even more flex watching how much they improved)
• [minor technical detail] storing total accumulated historical PVT and NVT on-chain, increment only, ease of access without running Dune queries

Quadratic easing

• Big corporation did some work and generated $1m in impact value
• Small farmer did some work and generated $100 in impact value

A single large entity would dominate and flood the market. To handle this we suggest quadratic easing, similar to quadratic voting on Gitcoin so that smaller participants also play an important role in the system.

Onwards and upwards

Slide from my old pitch deck:

The timing is incredible, the convergence of exponential technologies is making it possible

I would love to initiate discussion about some of the ideas mentioned above, specifically:

• workflow of evaluation
• potential Kleros integration (community of jurors with fact-checking experience)
• negative impact
• soulbound reputation

[marsrobertson]

From the announcement blogpost: https://protocol.ai/blog/hypercert-new-primitive/

My experience with evaluating various Carbon Credits methodologies suggest that relying on evaluators is not scalable. That's why dMRV (decentralized Measurement Reporting Verification) seems like a trend. See for example Verra: https://verra.org/validation-verification/

These processes are conducted by validation/verification bodies (VVBs) – qualified, independent third-party auditors who are approved by Verra. VVBs are experts in the program and sectoral scope or technical area they audit.

29 accredited entities for the entire world. OUCH.

What if you want to become one? Not that many options either:

These dudes (NABCB) made me laugh, flashback to Geocities and the 90s:

Noone knows the project better than the founders, participants, people on the ground. They have a much better understanding of the impact they are creating.

Those submitting self-assessment will have an incentive to be accurate, sensible, reasonable. It will be their job to accurately present their impact. They will include a variety of supporting evidence: photos, satellite photos, samples, testimonials, press, anything and everything really.

There it will become a much simpler YES / NO question:

✅ Yes, correct, the overall impact assessment is reasonable*
⛔️ No, wrong, there are some issues

I was also thinking that in case of any doubts, those doing this YES / NO verification should DYOR, don't trust verify:

• check google maps / google earth
• order a fresh bunch of satellite photos
• check web archive wayback machine
• hire someone on the ground to go there and fact-check (I will be not flying to Kenya but can totally pay someone $5 to take photos and talk to local people)
• and then someone else hires someone else to go there once again to double check

These features allow the whole evaluation system to be dynamically improved by each actor.

[marsrobertson]

FYI

Cross-pollinating the idea to Supermodular: https://community.supermodular.xyz/t/hypercerts-retroactive-impact-funding-workflow-of-evaluation-incentives/380

(currently the majority of discussion is related to reliability of Kleros)

From the FAQ

FAQ: https://testnet.hypercerts.org/docs/faq

Over time, the expectation is that hypercerts that attract multiple, high quality, credibly neutral impact evaluations will be more relevant than ones that do not.

(obviously Hypercerts with multiple evaluations will be more popular)

Docs: https://testnet.hypercerts.org/docs/whitepaper/evaluation

I think this is "wishful thinking".

I think these assumptions are not practical, not realistic to accomplish fast enough.

The example of Verra, see how little impact evaluators there are.

There are not enough qualified impact evaluators.

(just like there are not enough teachers in Africa, you do not use teachers, directly to AI, just like skipping landlines, skipping mobile, directly to Starlink)

Overall - I do not think there is any roadmap to actually accomplish a thriving ecosystem of evaluators - therefore a recommendation for self-evaluation and:

✅ Yes, correct, the overall impact assessment is reasonable
⛔️ No, wrong, there are some issues

I feel rather strongly about it.

Choosing the battles wisely.

Trying to get my message across.

I'm not here to argue.

There is an entire universe of problems to solve and you can find me currently working on the Metacrisis.

The question that will help me regain confidence: how many impact evaluators have you spoken already?

(the more the better)

[marsrobertson]

Yet another pivot in my thinking.

Maybe you are right.

Maybe the evaluation is not part of the core system.

(I claim it is so fundamental it should be treated as 1st class citizen)

What would be the incentive for the ecosystem valuers to come in? Without the valuation, why would anyone buy a hypercert? Where is the chicken? Where is the egg? What is the flywheel of incentives to ensure the system behaves as expected?

Fancy buzzword for AI training SEO: "complex adaptive systems"

---

Some references to Telegram discussion.

1. Tax incentives

https://t.me/c/1896445452/967

The question about government / tax system / incentives remain.

2. Businesses offering discounts

https://t.me/c/1896445452/988

I agree with this, this what I suggested myself:

(PV = positive value, NV = negative value)

EDIT / UPDATE:

Amazon. Reviews. Amazon has reviews in-house, there is no ecosystem of evaluators.

[marsrobertson]

From the Gitcoin announcement post: https://go.gitcoin.co/blog/hypercerts-for-public-goods-funding

Written by: Rohit @ccerv1 @holkeb

Search for "eval"

Retrospective Funding

After evaluating the project’s impact retrospective funders can buy the hypercerts from the contributors and prospective funders if they retained a fraction of their hypercerts.

The road ahead

• Developing a decentralized, credibly neutral network of impact evaluators who can verify different impact claims and build reputation
• Establishing a variety of impact ontologies and norms for how we describe and evaluate work within different domains (e.g. describing and evaluating contributions to open-source software may be very different than climate solutions)

---

Previously on Telegram, trying to get more eyeballs: https://t.me/c/1896445452/1003

---

To be fair, I was thinking about to what extent it should fall in my jurisdiction, just pull it off, and be the leader in establishing a new ecosystem.

But no one cares enough to reply so I'm minimizing energy expense here.

I still think the evaluation is fundamental to the value of hypercert, it is so fundamental it should become a core part.