Create Generic Signed Payload

[sdellava]

As quickly discussed, I'm happy to submit this very simple TIP that will allow to build a standard Selective Permanode INX plug-in

[lzpap]

Hey, @sdellava, couple remarks:

• messages are called blocks, as per TIP-24.
• Since this is an application level standard, it should not introduce a new payload type into the core protocol. You can use the Tagged Data Payload to store/publish your application specific data structure.
• Change the layer of the TIP to IRC, see the different layers in in TIP-1.
• Define the structure, layout and representation of the data you'd like to put into the Data field of the Tagged Payload.
• Explain the interface and components of the INX application you are planning to build.
• Explain how the INX application filters blocks, what syntactic/semantic checks should be implemented.
• Inlcude a rough API design on how the INX application exposes data through an API.
• Inlcude example payloads and their validation.

[lzpap]

Also, please update the PR folder structure to match the TIP process requirements.

[sdellava]

Thanks for your feedback.

In an initial evaluation I assumed that the public key could be used instead of the tag, however, it is evident that the tag allows different types of data contents produced by the same author to be specified. Therefore, I propose to create a new payload type (which I have tentatively denoted 6) in which the signature and public key are present in addition to the tag.

Signature and tags allow for the creation at the network level, thus with as little effort, i.e., as little power consumption as possible, of a simple communication protocol that I would call "lightweight stream channel." The tag can in fact contain the DID identifier of the recipient of the message, which can be encrypted with the private key contained in the DID document.

This payload format provides a very powerful tool for making low-cost data applications

In addition, the tag is useful for collecting and storing messages in a selective permanence INX plug-in, allowing the data consumer to collect data asynchronously.

I've removed reference to the INX plug in beacuse this block payload type is not solely useful for the realization of a selective permanode.

[alexsporn]

There is no real difference in computation overhead for a L2 solution to extract the signature from a protocol field or from the payload itself. I see no reason to pollute the L1 protocol with fields with signatures that it will not verify.

Explain to me the difference between this 2 usecases:

1. Current Tagged Data:

• Tag: myspecialpayload
• Data: mydatapayload+signature(mydatapayload)

2. The proposed solution

• Tag: myspecialpayload
• Data: mydatapayload
• Signature: signature(mydatapayload)

If the argument is, that Tag cannot be spammed with solution 2, then this is wrong, because I can also send invalid signatures and L2 will need to verify them anyway to filter these out.

[sdellava]

Defining a Layer 2 format that must use the Tag defined in the Layer 1 format to discriminate the meaning of the Layer 2 content is in my opinion much more wrong than creating a specific Layer 1.

Proper layering should not involve Tag at L1, but Tag, possible key and signature at L2. Or everything at L1.

True, there is no particular difference between the two options, but this is a motivation that reinforces this approach.

If nodes ignore the contents of blocks with payload type 5, they also ignore those with this payload type, so there is no side effect for the network.

Having this type of payload also makes it possible to create particular, very lightweight nodes that might instead ignore all other types of blocks and consider only those that have the special nature of not being retained by "normal" nodes.

Possibly, as an alternative and compromise, we can simply modify the Tagged Payload, adding after the date field, the key and signature as optional fields. In this way, a new payload format is not created but we reinforce payload type 5 by making it more robust and versatile.

[alexsporn]

Possibly, as an alternative and compromise, we can simply modify the Tagged Payload, adding after the date field, the key and signature as optional fields. In this way, a new payload format is not created but we reinforce payload type 5 by making it more robust and versatile.

Adding fixed optional fields does not make it more versatile. You have a clean canvas inside the Data payload where you can specify any fields for any use-case. Adding specific fields for special use-cases just pollutes the L1.

Currently you are thinking about 2 extra fields, public key and signature. But what happens then when the next requirement arrives and it wants to add a UUID to the data? Or a validity timerange? Or some reference to another previous block?

All this is application logic, not L1 logic, and thus should be handled on the application level.

If you want to standardise payloads for certain use-cases, to be interoperable with other applications, then that is what the IRC standards are for. Just look hat the current proposed NFT standards, it is exactly the same idea.

[sdellava]

The problem I see with creating a standard L2 format is that you basically need to define a also L2 payload type to allow the receiver to understand how to interpret the data contained in the L2 payload. In example, the singed data playload is type 0 and involves key and signature, while type 1 contains an encrypted message and thus contains key, signature, and public key of the recipient.

In other words, since there are no restrictions, the content of an L2 payload is ambiguous.

To resolve this ambiguity the only option I see is to make the use of the typed L2 payload mandatory for the L1 payload type 5. However, this requirement is equivalent to creating a new L1 payload type.

[sdellava]

To be clear and not to lose the focus of the problem: if we cannot be certain of the reliability of a Tagged Block, we cannot store it in a selective permanode.

The meaning of the payload is irrelevant to its preservation. The only thing that matters is that it is authentic, and the only way to verify this is to allow the sender to sign it, allowing the receiver to check the signature without ambiguity.

[sdellava]

Last consideration: the use of the network to transmit generic data is certainly a disturbing factor for high economic profile applications. However, this is IOTA's mission. Unless we are saying that this mission has changed, there is a need to provide IoT application creators with a reliable way to transmit and receive data on the new 2.0 network, even without having to commit economic resources. This seems to me to be the point of the L1 payload type 5, tagged payload. However, the new release of the network, and in general the amount of traffic the network will be able to handle, requires nodes to drop any non-essential blocks from their db as soon as possible, and blocks with Type 5 payloads are obviously not essential to confirm ledger status.

While I understand the contrariety of introducing additional functionality to layer 1, i.e., the communication layer between nodes, it seems to me that you are underestimating the fact that the proper reception of tagged data by a client through a plug-in is still a network function.

Selecting blocks with pyload type 5 based on the tag and storing them in the node to provide them asynchronously to clients who later request them is a network function even if it is realized through an extension of the node's functionality.

It is of course possible to make an application that evaluates the payload content in a proprietary way and decides how to handle it, but the point is exactly this: the missin of IOTA to support IoT applications should require that this problem be solved in a standard way, not in a proprietary way by each application.