proposal: metadata extensions

[lyoshenka]

LBRY has a base schema for claim metadata. This schema is very general. Different file types and communities will want to include metadata that is specific to them. It would be good to have a mechanism to suppor this.

One idea for this is to allow publishing of metadata schemas to LBRY itself. This would be a new claim type (Schema). The schema would be a protobuf file (or similar, if protobufs cannot be compiled live), plus other information that hubs need (e.g. which fields to index). Then a claim can reference a published schema to tell hubs how to decode extra metadata contained in the claim.

Schema claim type

• schema structure
• how to specify indexes
• schema versioning?
• can schemas be changed? that complicates things
• limit who can create schemas at first, for testing?

Stream and Channel claim changes

• add a way to add arbitrary data
• add a way to specify the schema for the arbitrary data

SDK search changes

The SDK would switch from using flags for search to using a simple DSL. This way the SDK does not need to know all possible metadata fields in advance.

• what DSL to use for search?
• how to reference fields in search?

Concerns

• this is an avenue for protocol users to inject code into hubs and/or control their behavior. It's possible to DDoS hubs or impose unsafe or expensive computations on them (eg indexing fields for search), esp in combination with user-created claims.
• i doubt that protobufs can be compiled on the fly (at least in Go). will we need another schema description language?

[lyoshenka]

we're doing a limited version of this here lbryio/lbry-sdk#3706

will revisit after thats been out in the wild