RFE: add support for multiple OpenPGP signatures per package

[pmatilai]

Initially requested in #189 and one possible implementation drafted in #1050, but lacking direction and motivation at the time. The topic rose again as in the context of Post Quantum signatures in #3363 - something rpm would rather not know anything about. #1050 added labels to each signature but this is apparently a goofy idea (I think I stole it from Debian at the time), and back then rpm v3 header+payload signatures complicated the backwards compatibility store quite a bit, which is where I ran out of steam in the face of lack of general interest. The discussion in #3363 provided us with a nice clear path ahead now, with multiple benefits, so much so that it'd be stupid not to do this now:

• support for multiple signatures has generic benefits and use-cases
• puts a further layer of insulation between rpm and crypto, something we have been actively driving for a couple of years now
• adds provisions for PQC without us getting directly involved in it
• fixes up a long-standing terminology mixup with tag names
• lines up nicely with the rpm v6 theme and timing
• most of it is already implemented

What the implementation will do:

• add a string array RPMTAG_OPENPGP tag and RPMSIGTAG_OPENPGP alias
• RPMTAG_OPENPGP may contain one or more independent OpenPGP signatures base64-encoded (the header doesn't support binary arrays)
• the RPMTAG_OPENPGP signatures are always header-only, and we'll call them RPM v6 signatures
• extend :pgpsig format to handle new format
• add the new tag to rpm -qi Signature: output
• rpmsign --addsign appends a new signature to the RPMTAG_OPENPGP tag
• rpmsign --delsign deletes all signatures from the package
• rpmsign --resign deletes all signatures before adding a new one
• verification will:
  • ignore unknown algorithms
  • require all known (and enabled) signatures to pass for a positive verification
• backwards compatibility is handled as follows:
  • when signing v4 packages and --rpmv6 is specified, the first added RSA/DSA/EcDSA signature is additionally stored in binary format to RPMTAG_RSAHEADER/RPMTAG_DSAHEADER as appropriate
  • v6 packages get only RPMTAG_OPENPGP signatures, unless --rpmv4 switch (added by the PR) is used - this allows rpm v4 to verify such packages, in which case it behaves the same as signing v4 packages

Allowing to replace or delete a specific signature would be out of the initial scope but can be done later. Additional controls for verification policies can/will be added later.

Edit: add default verification policy

[pmatilai]

@simo5 @nwalfield @mlschroe @ffesti thoughts - did I miss some finer details, are there elephants in the room etc?

[pmatilai]

One potential open question is --resign: right now it's just an alias for --addsign because there's no practical difference between the two. With this, it could mean something else, probably delete any existing signatures and then sign. Of course this just a minor non-critical detail and can be dealt with later as well.

[simo5]

Sounds reasonable that --resign will drop all signatures and add new ones.
I think the only potentially missing case here is the desire to drop only a specific signature.

The reason to do that is if you have a package with multiple signatures and you want to replace only one that had a signing key compromised while the others did not.

The use case is packages re-distributed by a 3rd party that wants to retain the original signatures and can't recreate them because they have no access to those keys.

I wonder if --resign could be enhanced to be able to specify a signature to replace, in which case it would only replace the specific signature and not drop them all ?

This is really a corner case and if it is complicated it can definitely be deferred or even not made available.

[pmatilai]

Yeah --delete and --resign deleting everything is basically just the simplest possible semantics to move forward. I certainly see use-cases for deleting or replacing a specific signature instead, and since that doesn't require any format changes it can be done later once the more critical stuff is out of the way. So I think if we move ahead with this plan, I'd just file another ticket for deleting/replacing a specific signature in some point in the future.

[DemiMarie]

I recommend against using base64. Just use length-prefixed binary blobs. If base64 is used, the decoder should be very strict and only accept data that round-trips correctly (so no whitespace, etc).

[pmatilai]

We don't want to add a "proprietary" format for such a thing because standard header APIs are then no longer usable for accessing and modifying it, and then you only have more code to worry about. We could also just use hex strings, size is not a concern here. But, if we can't be trusted to decode base64 then how are we expected to read the rest of the rpm? That's like being too afraid to leave the house because something bad might happen. I'll note that this is something that could be easily outsourced to rpm-sequoia.

Binary arrays could be sort of handled with existinging code u,sing an embedded header (it's just another binary blob afterall), using tag number as the index. But this gets weird and wacky and it's not like the header itself is a trivial structure to parse.

[pmatilai]

Folks wanting to discuss pros and cons of detached signatures are welcome to do so in the relevant topic but is off-topic here, and has now been flagged such. This ticket is about a new form of embedded signatures in rpm 6.0. Thank you.

[pmatilai]

One thing the description doesn't currently cover is the verbose level verification messages, in particular the enforcing mode where it spews out everything it looked at. For example with an unsigned package in enforcing mode, you'd get something like (the last two non-prefixed items stand for legacy Header+payload signatures):

/data/RPMS/hello-2.0-1.x86_64:
    Header RSA signature: NOTFOUND
    Header DSA signature: NOTFOUND
    Header SHA256 digest: OK
    Payload SHA256 digest: OK
    RSA signature: NOTFOUND
    DSA signature: NOTFOUND

I think we need to lump all the OpenPGP signatures under one label per range to make any sense out of this, ie:

/data/RPMS/hello-2.0-1.x86_64:
    Header OpenPGP signature: NOTFOUND
    Header SHA256 digest: OK
    Payload SHA256 digest: OK
    Header+payload OpenPGP signature: NOTFOUND

I'm tempted to add "Legacy" in front of the last item because that's what it is, and multiple signatures wont be supported for those. It's a dying breed already in v4, and I'm tempted to drop support for creating them at all in 6.0. We'll need to verify them to properly support v4 but we probably shouldn't even look for them in v6 packages. rpmsign will not create those entries for v6 packages anyhow, but it seems these days rpmsign is the last tool anybody uses for signing...

A possible sample output from a package with multiple signatures:

/tmp/hello-2.0-1.x86_64.rpm:
    Header OpenPGP V4 ECDSA/SHA512 signature, key fingerprint: e8a62c0512b06b5d2183ba207f1c21f95f65bbe8: OK
    Header OpenPGP V4 RSA/SHA512 signature, key ID 4344591e1964c5fc: NOKEY
    Header OpenPGP V4 EdDSA/SHA512 signature, key fingerprint: 152bb32fd9ca982797e835cfb0645aec757bf69e: OK
    Header SHA256 digest: OK
    Payload SHA256 digest: OK

[pmatilai]

Another related change is that we'll call header+payload signatures and digests "Legacy" from here on. Rpm v4 header signatures will be called legacy too but maybe not just yet.

[JanZerebecki]

But, if we can't be trusted to decode base64 then how are we expected to read the rest of the rpm? That's like being too afraid to leave the house because something bad might happen.

The point was not about the correctness of our implementation of base64, but that the format should have only one canonical encoding any alternate encodings being rejected. It also makes the format more reproducible.

When incorporating existing formats, it is suggested to use a format whose normal spec is strict in that regard. The reason is so that other implementations do not accidentally pick a common lax implementation, which cause different rpm implementations to accept and reject different rpms. Good formats have test suits to ensure alternative implementations reject anything besides the specified canonical encoding.

The same can happen with hex: 0xabcd 0xABCD encode the same. JSON is another example that does not specify a canonical encoding and this causes security bugs regularly.

One other of the advantages of only one canonical encoding is to prevent a MITM during transport of the rpm being able to modify it while the signature check would still not fail after the modification. While that may not directly pose a problem, this is a normal security precaution to be able to predict precisely what data exits on a system so that we can more easily verify it to be secure. Security bugs that are easier to exploit when arbitrary payloads are available are a regular occurrence on distributions using rpm.

There are additional ways we want the rpm format to only have one encoding: forbid more than one signature with the same key; reject if signatures do not appear in a canonical sort order (e.g. sorted by key fingerprint); each dnf/zypper/etc repo should come with a list of the exact keys whose signatures all need to be verified by rpm to be present (maybe keyed on packager and vendor?); etc.

[Conan-Kudo]

@pmatilai How do we decide when a package "fails" verification with multiple signatures? Would we have a policy tunable? Some kind of indicator as a "primary" signature? Or something else?

[simo5]

@Conan-Kudo the simplest policy is that signatures must all verify (why would you put multiple of them otherwise?).

The tricky part is how to handle signatures you do not understand, and I think the simplest policy, again, is to ignore those.

Note, I am not saying you should ignore signatures for which you do not have a public key, only signatures you do not have code for.

For sig where you do not have a key you need to get a key, just like for the one-sig case.

[JanZerebecki]

I disagree as explained above. If there is a sig you do not understand it should fail the rpm. If you do not have a key for one it should fail. Sigs for all specified keys need to always be present and to be in the correct order otherwise fail.

[simo5]

@JanZerebecki

This work is meant to create the conditions to move to new signatures over time while retaining backwards compatibility.

A draconian policy that does not contemplate the possibility of getting an RPM with unknown signatures would make any transition impossible. I am sure it should be an optional policy you can set on your system if you want to be strict, but we are talking about reasonable defaults here.

Of course if rpm does not recognize any signature it should fail, but as long as it can verify all known ones it is fine.

[JanZerebecki]

I would expect migration to happen this way:

• a dnf repo has only rpms with only one old signature
• dnf and rpm and distribution-repo-keys package or dns repo metadata for offering new keys are updated to support the new signature format and include the new key
• a release cycle happens so people can still update to this old repo (rolling distries wait time and make a snapshot)
• the next release of the repo is rebuilt to have all rpms only have new signature type with the new key; dnf knows old and new key, but the old repo only uses the old one and the new repo only used the new key; so the switch of a repo to a new key can happen atomically as long as dnf knows both keys
• after some time or a release cycle an update to distribution-repo-keys or dnf repo metadata with remove-key instructions removes the old key

Supporting two keys at the same time would be to hedge against one being compromised and migration would work the same as above.

I currently don't see how a more complicated migration would give you any advantages except more chance for errors.

If you really want to make it more complicated you could make a migration with phases of unsupported signature formats and/or optional keys by recording in the dnf repo meta data the list of keys of signatures to expect and marking some of these keys optional and/or skip-if-unsupported and passing that list to rpm. So you can have both flexibility and strict by default.

Does any of that work for your use cases?

If you have a use case that needs a more complicated migration, can you explain what leads to that need or what the advantages are?

[JanZerebecki]

(This implies different dnf repos with different key sets can coexist on the same system. So you could also add a new repo to an installation, stop updating the old repo, and at some point remove the old repo from this installation.)

[Conan-Kudo]

@Conan-Kudo the simplest policy is that signatures must all verify (why would you put multiple of them otherwise?).

Multiple signatures aren't necessarily for users installing to process, so it would make sense to ignore them in that case. For example, the signatures may be used to indicate something passed through certain stages. You may have a policy to validate them all, but it may not actually be a required policy. Some signatures may only be for some systems to validate but not others.

I can think of a variety of reasons for it. But regardless, I think it does make sense to have some way to indicate a primary/key signature to validate.

[JanZerebecki]

One important point of my suggestion is that the list of keys that are associated with a repo is signed and verifiable with the same list, but is distinct from the keys that are trusted to sign repos. Trusted keys are a superset of keys used in a repo.

This makes verifying parts of a repo more deterministic. If your config verified the repo, the uncompromised rpms will always verify. You do not suddenly get one rpm that has only a new signature that you always skipped on the other rpms because they had also an old one.

[pmatilai]

@pmatilai How do we decide when a package "fails" verification with multiple signatures? Would we have a policy tunable? Some kind of indicator as a "primary" signature? Or something else?

Hmm, I thought it was in the description as it's been discussed elsewhere but apparently not - will fix. The initial implementation will indeed simply require all signatures to pass. I expect us to have various extra controls later.

Rpm currently has disablers like RPMVSF_NORSAHEADER that operate on the tag level because that's how the signatures are spread out per algorithm, I think we'd extend this to simply operate on algorithm level instead, which means you can explicitly disable eg an algorithm considered compromised and if that's the only thing there was, you fail to get a positive verification.

As for unknown signatures, I hadn't really gotten there yet. But there is indeed only one possible default: to ignore anything unknown, because that's the only way to deal with forward compatibility - like @simo5 said. If in doubt, think about this: we add this new RPMTAG_OPENPGP signature tag into rpm now. Older rpm versions simply do not know about this tag, so they will not look there, much less try to verify anything in there. And that's exactly what allows forward compatibility to exist: older rpm versions can still verify the packages to the best of their abilities, we cannot expect them to do anything more. And that's exactly what we must do with the new signatures too - just ignore if not known. If there are no known signatures at all then you fail to get a positive verification, and that's again how it should be.

Note all the talk about positive verification: as a reminder, rpm 6.0 will ship with enforcing signature checking on by default (#1573). So you need to make that assumption when talking about this stuff now, otherwise none of it makes any sense. Just like rpm 4.x default signature behavior makes no sense whatsoever.

[pmatilai]

Also note that dnf (or otherwise) repositories are way out of scope and topic here, this is strictly an rpm level matter. How multiple signatures are dealt with on repo level is a repo tooling headache to be discussed elsewhere.

[pmatilai]

The point was not about the correctness of our implementation of base64, but that the format should have only one canonical encoding any alternate encodings being rejected. It also makes the format more reproducible.

When incorporating existing formats, it is suggested to use a format whose normal spec is strict in that regard.

If base64 is bad, what is good then? Plain hex better? @simo5 - thoughts on the encoding? I'm not particularly in love with base64, it's just a format we already have to deal with, and one that isn't as dumb as plain hex space-wise. For traditional signatures, space isn't critical because we're not expecting a single package to have hundreds of thousands of signatures. Are PQ signatures significantly bigger? (I've never seen one, I've no idea)

[simo5]

The encoding does not affect the feature for me so I have no opinion.
Some PQ signatures can be big.

The scheme that has the biggest signatures for now is SLH-DSA (formerly known as SPHINCS+) and the stronger variant has a signature size of 50KiB.

ML-DSA's (aka Dilithium) biggest signature is ~ 4600 bytes.