[VTS] 12/06/2024 Meeting notes

[Albermonte]

Wed 12/06/2024 there was a meeting in which the following points were discussed

• [VTS] Liveness score is not as precise as we would like to #23
• [VTS] Non active validators won't have size score #25
• [VTS] Size and Liveliness as they currently are calculated are inversely proportional to one another #22
• [VTS] Feedback from Community #24

The ideas gatherer from that meeting were the following:

Complement Reliability with Liveness

Liveness could be removed and included in the Reliability score.
Currently, Reliability only takes into account penalized / total blocks, regardless of whether those blocks were produced in epochs conituos or whether there are gaps between epochs.
Liveness is responsible of ensuring that a validator is always actively trying to get elected, but we cannot know this for past epochs due to technical limitations.

The new Reliability would also penalize validators for not being elected, this is a problem for small validators who are not elected often but since the idea of the VTS is mostly for pools, we believe that any serious pool should have a large enough initial investment to be constantly elected in, at least, 1 slot.

Liveness may not be important for VTS

Size already affects the liveness, with a good enough size you will get constantly elected.
The idea would be to modify the curve, making it so that small size validators would have a low Size score (because they won't get elected) but large validators with too much stake would also have a low Size score.
Draft example:

The problem with this idea is that we cannot know the size in past epochs, so it would be calculated only for the current epoch.

Geographical distribution

We think it is important to have equal geo-distribution and, more importantly, equal provider distribution.
But this is easily exploited by a bad actor setting up a lot of validators in a single provider or bypassed using tunneling.
One of the possible solutions to this would be taking into account Size of all validators in a single country, instead of total number of validators.

This will not be considered for now, we would need to think more about this and how to do that in a fair way.

Use slot asignation to calculate Size in the past

Since there's no way to know the stake a validator had in the past, the only fast and easy way would be to approximate it using the assigned slots in relation with the total slots (512).
We can get that data from the election blocks, it won't be as effective as the size but the relation assigned/total slots should be good enough for the Trust Score and can be used as a fallback.

Size curve changes

The current curve is designed to return 0 if the size value is >=25%, but we feel that this is too risky and no validator should have more than the 10% of the total stake in the network to consider the network as secure.

With this change, the curve was too harsh with small validators so we decided to increase the slope from 4 to 7.5. This change will allow small validators with <=5% to have a score of 1, until they exceed that 5% size, and then it will quickly drop to 0.

New curve graph

Non-paying Validators

We have no way to measure if the validator is or not paying, and more important, if it's paying what it should pay and to the address that should be paid.

Incentivize Validators to start before migration

There should be a way of assigning a higher score to validators that were part of the testnet before the migration. We had to discuss and come with ideas about how to do that properly.

The final conclusion was that no matter what we do, we will have to improve and change things in the future. We would have to take into account how many validators the network has, the amount of NIM staked and many other things that will cause future adjustments in the curves, parameters and factors used.

Almost all of the technical limitations would be solved if we use an archive node, but we want anybody to calculate the score at any time without using any third party entities. History node is the one we are using.

This are only the notes from the meeting, nothing was decided. Any opinion on any of the notes is highly welcome.

[joeleol]

Glad this is still in discussion, I generally agree with what is written - it has to be fair, and pleasing everyone will never happen.

I am still not sure I like the term 'Trust' for this scoring mechanism, it's more like a performance score - which is also fine for me, we have to distinguish between performant and non-performant validators.

Reliability is a very good starting point, as it is easily measured (is your uptime good, do you produce when you are supposed to, maybe even latency of block production / broadcast?)

I think for geographic distribution topic (which I am 100% behind), it's really tough, especially within Europe - my thought track on this leads towards an auditing/verification model to check that your hardware is where you say it is, but I feel this is not the direction that a community driven, open source and decentralised blockchain would want to go. It's just so easy to mask this, or change after an audit, think about people using public cloud / load balancers / proxies etc.

My last thoughts are that the bar is still set too low in terms of deposit to become a validator. I imagine you could filter a lot of undesirable behaviour / people messing around / 'gaming' the system / threats, by raising the stakes (or in this case 'skin in the game').

Finally, a question: Can history nodes be made to snapshot validator size once per epoch? I don't like the idea of back-calculating based on slots assigned, it's like measuring luck in a weighted lottery.

Keep going though, I think you're heading in the right direction!

[Albermonte]

I am still not sure I like the term 'Trust' for this scoring mechanism, it's more like a performance score

The term "Trust" is more for the end user than to represent what the score actually does. You are right, we are meassuring the performance and behaviour of the validators, but presenting it to the users as a way for them to know if they can trust a validator and delegate their NIM to them.

It's just so easy to mask this, or change after an audit, think about people using public cloud / load balancers / proxies etc.

That's why we are focusing on data that can be fetched from the blockchain and can be reproduced by any user at any time

Finally, a question: Can history nodes be made to snapshot validator size once per epoch?

That depends on the core team, but likely, it will cause too much noise and extra data that will grow over time.
The slots is not a final idea and will not (probably) have a big impact

[joeleol]

One additional thing, OG Validators (pre-migration) should get a gold star :)