Recovery of ID from previous uPort app

[trentlarson]

My group and I have been using uPort (ie. the app on our phones) and I'd like to convert us to these new tools. I just tried by testing out the export into a mnemonic seed and then an import, and I can do both of those operations inside Veramo with the entropyToMnemonic/mnemonicToEntropy functions inside the bip39 library. Export is easy by calling the entropyToMnemonic on the privateKeyHex on the identifier; import is a bit harder, but I think this is the right way:

import { entropyToMnemonic, mnemonicToEntropy } from 'bip39'
const EC = require('elliptic').ec
import { toEthereumAddress } from 'did-jwt'

const secp256k1 = new EC('secp256k1')
const DEFAULT_DID_PROVIDER = 'did:ethr'

    // inside my method
    const keyHex: string = mnemonicToEntropy(mnemonic)
    // returns a KeyPair from the elliptic.ec library
    const keyPair = secp256k1.keyFromPrivate(keyHex, 'hex')
    // this code is from did-provider-eth createIdentifier
    const publicHex = keyPair.getPublic('hex')
    const privateHex = keyPair.getPrivate('hex')
    const address = toEthereumAddress(publicHex)
    const newIdentifier: Omit<IIdentifier, 'provider'> = {
      did: DEFAULT_DID_PROVIDER + ':' + address,
      keys: [{
        kid: publicHex,
        kms: 'local',
        type: 'Secp256k1',
        publicKeyHex: publicHex,
        privateKeyHex: privateHex
      }],
      provider: DEFAULT_DID_PROVIDER,
      services: []
    }
    agent.didManagerImport(newIdentifier)

Unfortunately, this doesn't give me the same DID at the end.

Are there other ways to export from the mobile app? Can anyone comment on whether I've got problems in my import approach? I appreciate any help because I've got to find a way to migrate our IDs before I can start using this more extensively.

[trentlarson]

One interesting point: uPort generates a mnemonic seed of 12 words but the default I get from the entropyToMnemonic inside here is 24 words.

[mirceanis]

I think this is because the mnemonic is just a specific encoding of the entropy, meaning that if the entropy buffer is longer, you will get a longer mnemonic phrase from it.

• 128 bits of entropy -> 12 words
• 256 bits of entropy -> 24 words
• etc

[mirceanis]

There are several things to be aware of before attempting this migration. Unfortunately, it's not as straight forward as just migrating the private key of the DID. I will try to list some of the things that are required to deal with the uPort data.

1. The uPort DID & key derivation paths
   a) app-specific accounts
2. uPort data backup
3. A note on protocols

1. The uPort DID & key derivation paths

To get from the mnemonic to the base account private key, the simplest way is something like so:

import { HDNode } from '@ethersproject/hdnode'

const mnemonic = 'your mnemonic phrase here'
const UPORT_ROOT_DERIVATION_PATH = "m/7696500'/0'/0'/0'"

const hdnode = HDNode.fromMnemonic(mnemonic).derivePath(UPORT_ROOT_DERIVATION_PATH)
console.log(hdnode.privateKey)

//you can also quickly list the corresponding DID like so
console.log(`did:ethr:${hdnode.address}`)

Using bip32 / bip39 is also possible to get to the private key. Make sure to not confuse mnemonic, entropy, seed, and private keys.

import * as bip32 from 'bip32'
import * as bip39 from 'bip39'

const mnemonic = 'your mnemonic phrase here'
const UPORT_ROOT_DERIVATION_PATH = "m/7696500'/0'/0'/0'"

const seed = bip39.mnemonicToSeedSync(mnemonic)
const root = bip32.fromSeed(seed)
const node = root.derivePath(UPORT_ROOT_DERIVATION_PATH)
const privateKey = node.privateKey.toString("hex")
console.log("0x" + privateKey)

a) app-specific accounts

The uPort app also uses app-specific accounts.
This means that it doesn't use the same DID / ethereum address to interact with every dapp, for privacy reasons.

These other addresses / accounts can be derived using the same method from above, but changing the derivation path to the corresponding account number:

"m/7696500'/0'/1'/0'"
"m/7696500'/0'/2'/0'"
"m/7696500'/0'/3'/0'"

If you are using multiple identities then these sub-account paths will look something like:

"m/7696500'/1'/0'/0'"
"m/7696500'/1'/1'/0'"
"m/7696500'/1'/2'/0'"
"m/7696500'/1'/3'/0'"

2. uPort data backup

DIDs, claims, credentials, interactions with various dApps and messaging between DIDs, etc... are all local to the uPort mobile app. There is no central server that manages these, nor are they stored in a global state machine like the ethereum network.

There is an encrypted backup solution that you can enable from the mobile app but the protocol is very tightly coupled with the app architecture and it is not trivial to recover everything without using the app.

This means that if you are not using the uPort app, it is not enough to recover the private key of the root DID to recover all the credentials.

See also #202 about data synchronization between agents.

How backup works in the uPort app

The app is built using redux and redux-saga to store and manage global internal state, and it is this internal state and the events that mutate it that are encrypted and backed up.
For example, when you create a credential or when you receive a selective disclosure request, etc., these are represented as redux actions with plain object payloads, and it is this chain of events that are encrypted and sent to the backup service.
Later, when recovering, the events are downloaded, decrypted, and replayed locally to rebuild the same application state.
You can try to take a look at the backup saga to get an idea of how this is done.

If you want to recover all the credentials and interactions, you will have to recover the signing and encryption keys and then emulate the same calls to the backup service to rebuild the same state locally.

Then, you'll probably want to extract all the credentials from this state tree and store them in a veramo agent.
The credentials that uPort mobile was using are not 100% W3C compatible so they may require a new message parser plugin to be used in veramo.

3. A note on protocols

All of the above is just about the data.
If you wanted to recreate a system that can interop with the legacy uPort ecosystem you would also need to implement some of the protocols.
For example, selective disclosure in uPort is also assuming that an ethereum network will be used, and you can also make other kinds of requests that are not implemented in veramo (see uport messaging specs)

While this would not be particularly difficult to achieve using custom veramo plugins, it is not on our roadmap at the moment because of resource constraints and the fact that the veramo core should remain rather generic.

We can offer support only on a best effort basis, but I think it would be a testament to SSI if this kind of contributions came from the community of users.

[trentlarson]

Thank you for such a comprehensive reply! This is immensely helpful and has given me more options. Also: thanks for replying so quickly... I can't even express how much that helps maintain excitement. :-)

Unfortunately, I'm still not able to take my uPort mnemonic and recover the same address in Veramo.

Note that:

• I assume you meant to put node instead of hdNode in your first snippet. (fixed)
• The UPORT_ROOT_DERIVATION_PATH you gave is not a standard derivation for Ether (eg. the defaultPath in the ethers library). I assume that is the derivation used by uPort and it does not follow BIP44 (and I'll look at the source) but I'll keep testing with both just in case.
• One behavior that I haven't figured out: when I use only Veramo and export the identifier key with bip39.entropyToMnemonic(identifier.keys[0].privateKeyHex), then I use your second code snippet and then const publicHex = node.publicKey.toString('hex'); const address = didJwt.toEthereumAddress(publicHex); I don't get the same Ethereum address, even though that is all bip32/bip39 library code. (I can't find any other "...ToMnemonic" type of function in ethersproject or elsewhere so throw others at me and I'll try with the other code you posted.) That's not strictly related to this recovery issue, but it may give me ideas if you show me what I'm doing wrong.
• I may still be misunderstanding the relationship of those terms: I thought the entropy was used to create the mnemonic (despite the name 'mnemonicToEntropy' which implies that we can recover the original non-check-summed entropy which is impossible), the mnemonic-plus-password-derived-number == seed, and the private key is derived from the seed via bip32... corrections welcome.

Thanks for the background on the backup and the protocol. I feel those paths lead to more tygers so I'm inclined to keep working on the mnemonic (since I've successfully used that to restore into other uPort instances and that's what my group has used as backups). Maybe there's another way to export all the identifier data (or chain code, etc) from uPort... just brainstorming.

Anyway, as for the original issue: I haven't found how to use the 12 words from uPort and restore them into Veramo (ie. get the same DID). I'm gonna look at uPort source some more and try a few more combinations of code from what you've given me.

[mirceanis]

I assume you meant to put node instead of hdNode in your first snippet.

yes, thanks for spotting that, I corrected my reply

The UPORT_ROOT_DERIVATION_PATH you gave is not a standard derivation for Ether (eg. the defaultPath in the ethers library). I assume that is the derivation used by uPort and it does not follow BIP44 (and I'll look at the source) but I'll keep testing with both just in case.

Yes, it is not a BIP44 standard path. This was intentional since uPort was never intended as a coin wallet and we still strongly believe that identities should be separated.

I may still be misunderstanding the relationship of those terms: I thought the entropy was used to create the mnemonic (despite the name 'mnemonicToEntropy' which implies that we can recover the original non-check-summed entropy which is impossible), the mnemonic-plus-password-derived-number == seed, and the private key is derived from the seed via bip32... corrections welcome.

Yes, you got this right. The intended operations in bip39/32 are: entropy <-> mnemonic -> seed -> privateKey.

The mnemonic is an encoding of the entropy and it can be decoded as well, hence the existence of mnemonicToEntropy() methods.
Now, since the entropy buffer is expected to be random noise, it means that any input of the proper length is accepted, including a privateKey. This is not the intended use, but it is indeed possible:

console.log('0x' + privateKey.toString('hex'))
const keyAsPhrase = bip39.entropyToMnemonic(privateKey)
console.log(keyAsPhrase)
const recoveredKey = bip39.mnemonicToEntropy(keyAsPhrase)
console.log('0x' + recoveredKey.toString('hex'))

Be careful here, because the mnemonic you get when interpreting a private key as a mnemonic phrase is not usable with the proper bip39/bip32/HDNode pipeline. I would strongly recommend against encoding private keys this way since it will likely lead to confusion.

One behavior that I haven't figured out: when I use only Veramo and export the identifier key with bip39.entropyToMnemonic(identifier.keys[0].privateKeyHex), then I use your second code snippet and then const publicHex = node.publicKey.toString('hex'); const address = didJwt.toEthereumAddress(publicHex); I don't get the same Ethereum address, even though that is all bip32/bip39 library code. (I can't find any other "...ToMnemonic" type of function in ethersproject or elsewhere so throw others at me and I'll try with the other code you posted.) That's not strictly related to this recovery issue, but it may give me ideas if you show me what I'm doing wrong.

I think you are still confusing the operations that can be done with keys/mnemonics/seeds. The difference comes from trying to encode the private key as a mnemonic phrase (bip39.entropyToMnemonic(identifier.keys[0].privateKeyHex)) which leads to a mnemonic that is not usable with the default bip39/32 pipeine.

At the moment keys in Veramo are not generated from a mnemonic phrase using bip39/32 operations.
However, it should be possible to derive the private keys from other wallets that use mnemonics and use the private keys directly in Veramo, which is what I understood you wanted to do for the uPort app migration attempt.

The operations would be something along these lines:
uPort mnemonic -> HDNode derivation (first code snippet ) -> uPort private key -> import privateKey in veramo

I hope this helps clarify some things.

[trentlarson]

OK... fantastic!

First of all, thanks again for a comprehensive answer. I see now where I was making mistakes and mixing up concepts in my code (eg. a mnemonic from a private key).

But most importantly: I just tested your code again and it works! (I would have sworn I tried exactly what you said earlier and it didn't work before.) Woo hoo! I'm very excited to have a clear path to migrating my users. Thank you, @mirceanis !

[jasheal]

@trentlarson Note that Veramo by default stores private keys in the database. In RN this is not encrypted or in a secure enclave (beyond manufacturers default device encryption) so ideally you would need create a HD identity plugin that creates and imports HD keys and stores them in the secure enclave. We have looked in using React Native KeyChain for this. It is something on our horizon but we don't have a specific timeline for it yet however if you decide to build this plugin, we of course could provide help / guidance with as it would be a valuable contribution.

[trentlarson]

I love that idea! Now it'll be stuck in the back of my mind. :-)