Standardizing of HD wallet derivation paths (BIP32, BIP39, BIP44)
coder5876 opened this issue · 75 comments
References:
We are now seeing a few different Ethereum HD wallet implementations with different HD derivation paths.
When we wrote Lightwallet I considered using the BIP44 specification defined by
m / purpose' / coin_type' / account' / change / address_index
which gives a default path of m/44'/60'/0'/0 for Ethereum. I felt at the time that this was very UTXO-coin specific and didn't make that much sense for Ethereum, so I had a scheme that was based on having different paths for different keys (like signing keys, encryption keys etc) and different identities, and the focus was not on sending/receiving Ether. The default path became m/0'/0'/0' (Purpose/ID index/key_type), but we included the ability to specify a path of your choice.
The Jaxx wallet used lightwallet (at first) under the hood and chose an HD path of m/44'/60/0' which is almost BIP44, except the change path is not used.
There is this BIP44 HD wallet
https://github.com/trapp/ethereum-bip44
by @trapp which uses the full BIP44 path m/44'/60'/0'/0.
@axic created an HD wallet library which is used in ethereumjs/testrpc here:
https://github.com/ethereumjs/testrpc/pull/44/files#diff-f3d2a8282458e5cf231eee263cd57075R32
This also uses the full BIP44 path m/44'/60'/0'/0.
UPDATE: Clarification: The ethereumjs-wallet library from @axic does not impose any path, rather that when integrated in ethereumjs/testrpc the path used is the standard BIP44 one.
So in the spirit of trying to have some interoperability between HD wallets I would ask for some input:
- Should we just settle on the full BIP44 path
m/44'/60'/0'/0(which seems to be the most popular based on my unscientific sample)? - Does it matter that
changedoesn't make much sense for an account-based architecture like Ethereum?
For Exodus, we use m/44'/60'/0'/0/0, we'll probably just stick with this since it's a solved problem. Please let me know if I'm missing anything.
@jprichardson Your path refers to the first address created, right? So in general m/44'/60'/0'/0/n for the nth wallet created. So your project is another data point favoring the standard BIP44 path.
From testing the Jaxx Chrome plugin I can verify that Jaxx uses the HD root m/44'/60'/0' which makes the single derived Ether address m/44'/60'/0'/0. So this is not really compliant with BIP44. Would be nice to have Jaxx on board with the standard since it's a high-profile wallet.
I believe @chrisforrester is currently working on full HD-support for Ether, will you be using the full HD path when this is released?
For the record, my library is at https://github.com/axic/ethereumjs-wallet and it is used in a few places, not only testrpc, however it doesn't impose any path restrictions, in fact it doesn't have any path in the source code as it is only a toolkit to build wallets.
I have used the BIP44 path on Quorum just for the reasoning that should work well with current Bitcoin wallets. It turned out Trezor itself doesn't have any restrictions either. No idea about the others.
@christianlundkvist I understand your point and I think I'm in favour of that reasoning, with the exception of different number for purpose. I simply don't think 0' is a good one. Maybe it would be useful fleshing out a few use cases and how those fit into your scheme.
@axic Yes, I should have been more clear that your library is completely agnostic, I updated the original post with this.
Personally I think I was overthinking things a bit in my HD path reasoning and that it's too soon to start looking for some optimal HD scheme that will cover all cases. Sticking with BIP44 makes sense at this early stage I think since most use cases at this point are about sending/receiving Ether.
It might be early to make a decision, nevertheless can you describe your scheme in depth? It can be a good starting point.
@axic Sure, it was not really that advanced: The main idea is that you can have one seed for different identities or Personas. For persona nr n (at the path n') you have one branch for Ethereum signing keys m/0'/n'/0', one branch for keys to be used for asymmetric encryption m/0'/n'/1', one branch to be used for symmetric encryption m/0'/n'/2'. That's about as far as I got 😄
I was also thinking about separating signing keys used to mainly hold Ether with keys mainly used to interact with smart contracts, so in this case I would probably use something like BIP44 for holding and transacting Ether and the above paths for interacting with contracts.
@christianlundkvist Hi everyone :) Chris here.
I'm deep into the ethereum HD side of things. As Christian mentioned, our existing Jaxx project only uses the root conventional "account" node, which is obviously "wrong", however given the nature of the contract ecosystem, I don't think the bitcoin send + change really applies.
A lot of this is user expectation handling. Obviously privacy for conventional transfers of ether around the network are a good use-case, and for this I'm using the following standard:
m/44'/60'/0' is the "account" node.
m/44'/60'/0'/0/x is the "receive" node.
m/44'/60'/0'/1/y is the "change" node.
If the user receives ether into the current public receive node index, the x index will increase.
Sending ether from the wallet, I'm sorting through all the receive nodes, seeing if any specific node has the balance to cover the transaction, and sending directly as a single TX from that node. any change, gas price differences etc, will be returned to this address.
In the event that this highest-balance node cannot cover the send request cost, I'm appending tx together into a batch. Obviously the tx cost increases simply from the extra gas cost required. If requested, I'm going to build some sort of smarty-pants optimizer to give people some options with this (tx completion time to cost and so on)
To give users their total "spendable" balance, I take a look at the address they're trying to send to, calculate the total gas cost per tx, and subtract those from the user's "spendable" amount.
Any nodes that don't have the ability to send 1 + 21000x50Gwei are below the finite dust limit, and so aren't taken into account. They still have minor utility however as they can be swept completely if they had another deposit into that specific node.
The other option I considered was the complete reverse, always packaging the smallest amount account into an internal "change" node and sending from there. benefits to the user would be that anyone seeing a tx from them would always have a single tx on their list from the "change" node address.
However, this will cost gas to do, and has a delay in the case where a user receives a few tx at nearly the same block, and wants to send out immediately.
So I'm not doing this "proactive" packaging, instead I'm doing a "lazy" methodology. It should work fairly well.
As I need to support previous users, I'm going to apply that original node balance (yes I should have done a bit more thought about that earlier on :D) optionally to the first "receive" node index. Otherwise it'll be kept as a virtual node and used in that "lazy" passthrough system. Our existing pipeline includes transferring funds from an ethereum paper wallet, and this should be a rational analog to that system.
My thoughts on where to go from here re: contract watch addresses, contract ownership accounts, are that people are also going to want token support, so the UI is really the big undetermined factor right now. Since Mist has a basic support for this there's no huge rush, and since this process will form the expectation of how users go about accessing their accounts it's best to do this correctly instead of hacking something together that will have major ramifications in the next year as onboarding continues.
@chrisforrester Thanks for the detailed run-through! So it seems most HD wallets are now using the full BIP44 path. Support for tokens are a good question, I have some thoughts about which paths to use for that, but that's for another EIP. 😃
I was thinking that we could extend the metaphor logically.. we could use
m'/60000'->70000' or the like.
On Sat, Mar 26, 2016 at 2:12 PM, Christian Lundkvist <
notifications@github.com> wrote:
@chrisforrester https://github.com/chrisforrester Thanks for the
detailed run-through! So it seems most HD wallets are now using the full
BIP44 path. Support for tokens are a good question, I have some thoughts
about which paths to use for that, but that's for another EIP. [image:
😃]—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#84 (comment)
linking SLIP44 (used by jaxx) which proposes constants for different coins.
That said, is there a requirement for the branches to be numbers? seems like using a name would work just as well: "ethereum" or "eth"
BIP32 is the one defining the derivation:
ser32(i): serialize a 32-bit unsigned integer i as a 4-byte sequence, most significant byte first.
and see the usage for ser32.
Additionally the private-to-private derivation (hardened key) has a condition to only accept i ≥ 2^31. If you would use your text encoded as hex, that would always be a hardened derivation.
Ethereum doesn't have change addresses and, realistically, BIP32 is far from ideal for Ethereum. That said, its not a bad bridge until Serenity.
I would like to +1 the idea of only using hardened addresses. This facilitates privacy by default and requires that a developer or user manually do something in order to disable privacy. Moreover, the linkable nature of non-hardened addresses is out-of-scope for ethereum – with bitcoin it makes sense because there are change addresses. With ethereum, and especially with smart contracts, the feature is moot.
I'd like to bring up an interesting use-case.
(caution, my absorption of this is still quite new, someone please correct me if I'm wrong :D )
If I use non-hardened addresses, I can derive all of the public keys from the receive node's extended path.
(caution ended, the stuff below is legit)
Using this, as I sometimes have to batch transactions together, I can create a method of determining if, from any given one address, another address is associated with that batch, as long as they came from the same HD wallet.
In the case where this is desired, a user can send an "audit requestable" hd batch that have some data in them relating to a common nature(# of transactions in a batch, extended public key, username), that will make it really easy for the receiver to be able to hook into our one-name integration and say "yep all these transactions came from dudewiththehatnonotthatdudetheotherdude" and even then, be able to in the UI batch these as a single item.
and even better, if I hook this into the ethereum contract mechanism, I can have a way of batching transactions together to be "counted" as coming from a single source by contracts, and also allow, optionally, easy auditing of bank activity.
Whats then the advantage of using HD wallet here at all? Why then not reusing addresses?
This sending multiple tx from many accounts with dust, doesn't seem smart of efficient to me.
[offtopic]
btw does jaxx now finally provide enough gas? we had a lot of users who can't send from jaxx to kryptokit to a wallet contract..
[/offtopic]
advanced mode for ethereum contract support has been out for a few weeks :)
you're partially correct. it's not super efficient if you have to batch a bunch of small transactions. it's also not super efficient on the BTC side of things either since you have to pay for the data itself in the uxto transaction.
what you're giving up for efficiency is privacy. I'd like to make it as private as you'd desire it to be.
having a single address and then sending that out for mixing etc. involves a ton of middlemen, and I'm not sure of the logistics of the payment routing to keep privacy. immediately you can see that you've sent to a mixing service, and if you ever don't do that, and one of the end points leaks information about your origin address, you've lost all privacy anyhow, and have paid for the efficiency loss the entire time.
if part of an HD wallet becomes compromised re: privacy, I should only be exposing (with the user's opt-in! and when the network is more understood, opt-out, maybe.) the batched transactions themselves.
So we're coming to the end of the Ethereum HD development, and I'm going to use the recommended pathing. While there is a risk of undermining the entire account chain if they lose a private key from one of the HD nodes, it won't go below that, so our Ethereum and Bitcoin sides (along with whatever else coin-type we'll implement using the HD schema) are isolated.
The main reason is the hardware wallet functionalities. Though there aren't any Ethereum compatible models on the market that I'm aware of, I'm building our product to take their existence-in-the-future into account.
@chrisforrester OK great! Looks like it's BIP44 all around then! 😄
While there is a risk of undermining the entire account chain if they lose a private key from one of the HD nodes
So I didn't quite understand the "audit requestable" HD branch idea, does it require you to share the master pub key for the account chain?
I'm going to write up a proposal for HD paths for Ethereum tokens now also.
Given the nature of the privacy concerns, I'm thinking that the audit feature would be such that if you desired, you could encode the:
(for batchItemAddress1) "sha3(batchItemAddress2 + batchItemAddress3 + batchCodeGeneratedFromTheApp)" into the data field for the transactions.
then if you'd like to have an audit done, you'd provide the batchCodeGeneratedFromTheApp and the three addresses from the transaction batch to the auditor, so they could confirm that each of the transactions that were marked by this conformed to the other info in the batch code (things like a salt, and a block + block hash to do a proof of existence).
@christianlundkvist @chrisforrester
What is being proposed? A few thoughts...
Foremost, the linked BIPs are for Bitcoin and the design of the data structure is built around the idea of change addresses. Without change addresses, what is the point, and why not use a KDF?
To be more specific, and as noted by @frozeman, Ethereum choose not to adopt an HD wallet scheme. Two reasons, 1) the hashing/kdf scheme is different than anything else in ethereum, and 2) we have contracts. Contracts can send and receive multiple transactions which makes change addresses moot. For example, if you create Address /0/0 and /0/1 and then derive a contract from /0/0 and a contract from /0/1, the contract addresses don't follow the HD scheme and their transactions aren't linkable.
So in the spirit of trying to have some interoperability between HD wallets I would ask for some input:
What interoperability? Its one thing to extend Bitcoins wallet standard to "support" interoperating with Ethereum. Its different to implicitly extend the Bitcoin wallet standard to be an Etheruem standard. Instead, if I can make a recommendation, it would be to propose an Ethereum wallet standard along with an interoperability standard.
All together, Bitcoin's HD wallet standard is incompatible with Ethereum and is specific both to Bitcoin and the secp256k1 curve. Ethereum applications need more than what's proposed here. I don't mean to hinder progress or dismiss a proposal, and I'm happy to submit solutions, however I think its worth discussing whether BIP32/39/44 should be used to begin with.
@subtly with Jaxx, we're not using change addresses, everything is running off of the receive node path. Interoperability standard is on the context of the node derivation and the ability to create extended nodes for only being able to generate the public addresses, ethereum wallet standard is being able to have the same context of enhanced privacy for transactions.
maybe there could be a benefit from writing a mixer inside of a contract, and having everything funnelling through there, but AML regulations and the like are quite real. No company operating in areas where proactive AML practices are the the way things need to be done can really write a mixer.
in association to that, even if it was cheap enough to run the contract itself, which I haven't seen any hard evidence for, it's not like you can do it with the entirety being visible on the blockchain, without something like ring signatures, which have their own lower-bounds on how many users are required to make things actually not stall out while waiting to fill enough of a mix order.
http://ethereum.stackexchange.com/questions/387/is-it-possible-to-achieve-anonymous-mixing-through-the-ethereum-protocol
https://www.reddit.com/r/ethereum/comments/41m5ox/ethereum_mixer/
in regards to your claims of incompatibility.. really not so. using the exact same node structure:
for private keys:
bitcoin:
tempPair[0] = this.getPrivateKey(false, lastIndex + 1).toWIF();
ethereum:
tempPair[0] = this.getPrivateKey(false, lastIndex).d.toBuffer(32).toString('hex');
with "getPrivateKey" basically being:
HDWalletPouch._derive(this._receiveNode, index, false).keyPair;
and for addresses:
HDWalletPouch.getCoinAddress = function(coinType, node) {
if (coinType === COIN_BITCOIN) {
var pubKey = node.keyPair.getPublicKeyBuffer();
var pubKeyHash = thirdparty.bitcoin.crypto.hash160(pubKey);
var payload = new Buffer(21);
payload.writeUInt8(node.keyPair.network.pubKeyHash, 0);
pubKeyHash.copy(payload, 1);
var address = thirdparty.bs58check.encode(payload);
return address;
} else if (coinType === COIN_ETHEREUM) {
var ethKeyPair = node.keyPair;
ethKeyPair.compressed = false;
var ethKeyPairPublicKey = ethKeyPair.getPublicKeyBuffer();
var pubKeyHexEth = ethKeyPairPublicKey.toString('hex').slice(2);
var pubKeyWordArrayEth = thirdparty.CryptoJS.enc.Hex.parse(pubKeyHexEth);
var hashEth = thirdparty.CryptoJS.SHA3(pubKeyWordArrayEth, { outputLength: 256 });
var addressEth = hashEth.toString(thirdparty.CryptoJS.enc.Hex).slice(24);
return "0x" + addressEth;
}
}
BIP32 specifies compressed keys and, instead of simply deriving the pubk from a keypath, the posted code has to derive the public key from a secret. Thus, I see this proposal as extending BIP32 which makes it a BIP not an EIP. Issue #85 is a good example as to why BIP32 simply isn't sufficient for ethereum – having to do 5 KDF operations for interop is redundant, a hack, and also a BIP32 extension.
Not sure how Jaxx is relevant; I'm also using BIP32 but I'd rather see us develop an improvement rather than adopting the status quo.
If there isn't interest in discussing an improvement then please concisely state the proposal, e.g. that BIP32 is adopted, the purpose, and supporting use-case as to why a scheme which supports CKDpub((Kpar, cpar), i) → (Ki, ci) is a necessity for Ethereum (vs using contract code or privacy-by-default).
Please do not discuss AML here. This isn't R3 or HyperLedger.
/cc @chrisforrester @frozeman @vbuterin @alexvandesande @obscuren @Gustav-Simonsson @gavofyork @christianlundkvist @fjl @wanderer (CC'ing devs involved in secure key storage scheme used by official Ethereum clients)
honestly, your tone is disparaging and I haven't heard from you about the
other options I said would be possible but are completely unproven.
basically, my thought is such.. either we follow how bitcoin does stuff, or
we'll end up building something that if we desire interoperability, we'll
have to backport it back to bitcoin. then we'll have the same complaint
from the other side.
and yes, I'll discuss AML if I feel it's appropriate. whatever fantasies
you may be engaged in have to be grounded into the real world at some
point, and explicitly building something otherwise in a country that has
these laws in effect, is about a step and a half away from being shut down.
I understand that you may not feel it's a concern, but a lot of other
people do and when it comes down to it, it's about integrating these
blockchain technologies with the real world, not the other way around.
regardless of anything else, having something running client side that
provides
- better privacy in the general case
- the ability to reduce that privacy for outside inspection
- no additional contract cost
- no additional contract storage
- the ability to regenerate in a deterministic way a bunch of individual
subaccounts - said deterministic regeneration also working for different core projects
seems like a win to me. I'm done having this pedantic conversation, you've
made your point and I don't believe there's any substance there.
On Sat, Apr 2, 2016 at 3:33 PM, subtly notifications@github.com wrote:
BIP32 specifies compressed keys and, instead of simply deriving the pubk
from a keypath, the posted code has to derive the public key from a secret.
Thus, I see this proposal as extending BIP32 which makes it a BIP not an
EIP. Issue #85 #85 is also a
perfect example as to why BIP32 simply isn't sufficient for ethereum
– having to do 5 KDF operations for interop is redundant, a hack, and also
a BIP32 extension.Not sure how Jaxx is relevant; I'm also using BIP32 but I'd rather see us
develop an improvement rather than adopting the status quo.If there isn't interest in discussing an improvement then please concisely
state the proposal, e.g. that BIP32 is adopted, the purpose, and supporting
use-case as to why a scheme which supports CKDpub((Kpar, cpar), i) → (Ki,
ci) is a necessity for Ethereum (vs using contract code or
privacy-by-default).Please do not discuss AML here. This is Ethereum, not R3 or HyperLedger.
/cc @chrisforrester https://github.com/chrisforrester @frozeman
https://github.com/frozeman @vbuterin https://github.com/vbuterin
@alexvandesande https://github.com/alexvandesande @obscuren
https://github.com/obscuren @Gustav-Simonsson
https://github.com/Gustav-Simonsson @gavofyork
https://github.com/gavofyork @christianlundkvist
https://github.com/christianlundkvist @fjl https://github.com/fjl
(CC'ing devs involved in secure key storage scheme used by official
Ethereum clients)—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#84 (comment)
Thanks for the input @subtly. I totally agree that Ethereum key management in general is a huge question that is bigger than Bitcoin and will not be solved in the short term by BIP32, BIP44 or any other BIP. Also agree that change addresses are a feature of UTXO-based blockchains and are not intrinsic to Ethereum (hence why I initially chose a different HD path for Lightwallet).
This was meant simply as a proposal that attempts to introduce a short-term suggested scheme for the limited case of Ether wallets that currently use BIP32 HD derivation. Several such wallets already exists and there are good reasons for using HD schemes. For instance, HD wallets allow cold wallet/payment solutions where you can export a master public key, safely generate an unlimited amount of unique payment addresses on an online computer and have payments go directly into cold storage. This is an example of things that are being built right now. Furthermore there are many mature bitcoin HD libraries that can be and have been repurposed for Ether wallets.
Standardization might be a bit of a misnomer here, since I'm not trying to impose a key management solution for general Ethereum interactions or stifle experimentation. This is merely a suggestion that if you have a wallet focused exclusively on sending/receiving/holding Ether, then this is a HD path that you can use that other wallets are also using. I fully expect other innovative key management solutions to arrive, and I've been thinking a lot on this topic myself.
I'm also using BIP32 but I'd rather see us develop an improvement rather than adopting the status quo.
Wholeheartedly agree that developing an improvement is necessary long-term, and I'm interested in any ideas you might have. However, until everyone agrees on such an improved system this gives a concrete suggestion that one can use today, with mature (bitcoin) libraries.
@christianlundkvist
If I understand correctly the proposal is to, short-term, establish that BIP32 is used as a KDF for generating ethereum addresses because A) it exists, and B) its being used. BIP32 and BIP44 specify a scheme which imply certain functionality and that's why I think this proposal is a BIP extension and not an EIP or Ethereum issue. In any case, the majority of "clients" are likely to be Mist or exchanges and whether or not BIP32 is being "used" is subjective and unlikely – so I really think the best course of action is discussing and writing an EIP rather than a smattering of bitcoin. I have nothing against Bitcoin or BIP32 but we can all agree Ethereum isn't Bitcoin.
CKDpub (BIP32) is a very sharp knife and I strongly discourage its adoption as a standard without clearly notating implications, use cases, and specifications (usability is important too).
Prior to this proposal, accounts aren't traceable and users expect this – they can move funds via sweeps or have the wallet export secrets. There is interoperability in that core clients use an agreed upon KDF scheme (found here: https://github.com/ethereum/wiki/wiki/Web3-Secret-Storage-Definition). If this particular proposal were accepted there would be an expectation of interoperability via BIP32 – hence, why I inquire as to specifications for interoperability and not a temporary solution.
@chrisforrester I don't mean to be disparaging. I simply don't see BIP32 as an improvement; I've used BIP32 and I know it can makes things easier however I'm interested to discuss a proposal which addresses interoperability and clear specifications which improve upon ethereum and gives us flexibility going forward so we're not stuck w/BIP32. The idea of opt-in audibility is laudable, though, I'm not sure how that is in support of BIP32.
Advantages of BIP32:
Existing libraries, adds advanced public key derivation functionality, provides hierarchical kdf. Facilitates further advanced functionality such as BIP45. (aka traceable and read only wallets)
Disadvantages of BIP32:
Not used by ethereum clients. Public keys can be derived from other public keys; Ethereum doesn't have UTXO transactions and can't utilise changes addresses which makes this feature moot. As pointed out, a user must also sign two transactions to send part of a balance to one address and part of a balance to a second address – this doubles the cost and the second transaction may not end up in the same block. BIP32 adds complexity, affects user privacy, and increases transaction costs.
Can we get these features out of ethereum? Yes.
Audits: Use ethereum contracts
Read-Only: Use CKDpub directly w/hardened parent pubk and no change addresses
Read-Only: Use mixer contracts
Read-Only: Use BIP32 as extension
As I understand issue #84, the use-case is "creating public keys via KDF" and not to utilize the derivation features provided by BIP32. (@christianlundkvist please correct me if I'm wrong)
What specific use cases is this proposal intended to address?
Can a simple KDF be used instead? Why / Why not? ex: sha3(secret | rlpList(...keypath...))
Are change addresses used, and if so, why and for what use cases?
as a client implementor ( https://metamask.io ) I mostly want a standard for seed+KDF
an aside:
Ethereum doesn't have UTXO transactions and can't utilise changes addresses which makes this feature moot. As pointed out, a user must also sign two transactions to send part of a balance to one address and part of a balance to a second address – this doubles the cost and the second transaction may not end up in the same block
this can be remedied by a generic tx de-batching contract, made possible by DELEGATE_CALL, though I don't know if anyone has built one yet
As pointed out, a user must also sign two transactions to send part of a balance to one address and part of a balance to a second address – this doubles the cost and the second transaction may not end up in the same block
Or do the Ethereum way and use a simple contract. What are you trying to accomplish?
On Apr 9, 2016, at 23:36, kumavis notifications@github.com wrote:
As pointed out, a user must also sign two transactions to send part of a balance to one address and part of a balance to a second address – this doubles the cost and the second transaction may not end up in the same block
Some input from the Coinomi wallet. In our account based coins we use the following BIP32/44 path:
m/44'/coin_type'/account'
That private key is used directly without further derivations or processing. This makes things simple as only a single private key is needed per account (and many accounts can be created from a single master key).
For wallets that use the m/44'/60'/account'/0/0 private key, what is the use case of the extra keys that can be derived? Is there a use case of an account having several associated private keys?
Update: we are going to use the path m/44'/coin_type'/account'/0 for the account private key.
Just a quick update, we've released our Ethereum HD update for Jaxx (0.0.17->current). The backend code is up on our website at jaxx.io, upper right "view source".
@psionic81 thanks for that. Cross compatibility is important to us.
If I understand correctly the proposal is to, short-term, establish that BIP32 is used as a KDF for generating ethereum addresses because A) it exists, and B) its being used.
More like: Since BIP32 is being used we suggest a particular derivation path for people who are using it.
so I really think the best course of action is discussing and writing an EIP rather than a smattering of bitcoin.
You are of course free to do this if you wish. My view is that we still have a lot of experimentation in front of us in terms of key management so we will probably see many variations of key derivations.
Prior to this proposal, accounts aren't traceable and users expect this
Accounts are just as traceable as with BIP32 as long as you don't share master public keys.
There is interoperability in that core clients use an agreed upon KDF scheme
That KDF is to derive a key from a password to protect the randomly generate key which is not what I would call a "KDF scheme". The default model is now "randomly create keys"
Public keys can be derived from other public keys; Ethereum doesn't have UTXO transactions and can't utilise changes addresses which makes this feature moot.
Change addresses is not the main use case of public derivation IMO. The main use case is being able to derive public keys on a non-trusted computer, for instance in a cold wallet or web payment scheme. Admittedly this is a special use case so may not be that valuable for general wallets.
What specific use cases is this proposal intended to address?
Mostly just suggesting a derivation path if people are using BIP32 libraries. Again, I'm not suggesting that everyone has to use this. Many wallets use BIP32, it's unnecessary for them to all be using different derivation paths.
Can a simple KDF be used instead? Why / Why not? ex: sha3(secret | rlpList(...keypath...))
Sure it can. It's simpler and easier to reason about. And for tokens it's a lot more elegant.
Are change addresses used, and if so, why and for what use cases?
Nope.
For wallets that use the m/44'/60'/account'/0/0 private key, what is the use case of the extra keys that can be derived? Is there a use case of an account having several associated private keys?
Yes, you can easily imagine someone wanting several addresses to split up tokens. Also I intend on using different keys for different purposes (like signing & encryption).
@christianlundkvist after some consideration we will release our Electrum integration with the path:
m/44'/60'/0'/0
It will use that private key directly and there will be a single address per account. Like this we can use m/44'/60'/0'/x to create additional keys for future expansion.
Unless there is an objection, I propose to submit a pull request to standardize this behavior in the BIP44 spec.
Jaxx's ethereum HD nodes use:
m/44'/60'/0'/0/x
but the original implementation did use
m/44'/60'/0'/0
directly. unless it's HD w/ different receiving addresses, using the latter is fine with me.
of note: electrum is android compatible.. unless you figured out where that determinism issue occurred there is a small but real chance that people that use it and then switch phones could be affected.
@erasmospunk So it seems as though Electrum and Jaxx will use different BIP44 paths then. Does Electrum use BIP39 for key derivation? I seem to remember this was not the case. In that case these wallets are not compatible anyway so I'm fine with whatever path is chosen.
@christianlundkvist Electrum uses BIP32 for key derivation but their recovery phrases are custom, not BIP39 (may they have the option to import BIP39 phrases). We (Coinomi) use BIP32/BIP39/BIP44.
If we wanted to support multiple addresses per account, we could use the m/44'/60'/0'/1/x path.
So "60" as the coin type part of the address means Ether. But how to differentiate between "Homestead Ether", "Ethereum Classic Ether", "Morden Ether" etc.? They need distinct IDs so it's clear you shouldn't use the same addresses on different networks (which would open the door to replay attacks)
Ether is 60.
Ether Classic is 61.
Morden is 1 because all testnets are 1.
At MetaMask we use m/44'/60'/0'/0 for BIP32.
https://github.com/MetaMask/metamask-plugin/blob/a245fb7d22a5fe08c4fc8c2c1c64d406805018a8/app/scripts/keyrings/hd.js#L10
we dont currently differentiate between network, but may do that in the future
we use npm modules ethereumjs-wallet/hdkey and bip39 to handle seed phrases and HD keys.
https://github.com/MetaMask/metamask-plugin/blob/a245fb7d22a5fe08c4fc8c2c1c64d406805018a8/app/scripts/keyrings/hd.js#L2-L3
Note! There is an unfixed bug in lightwallet (and bitcore) libraries that misgenerates 1/256 bip32 keys.
What is the current status of this? I can't quite tell if there is a consensus on this or not based on the thread. It sounds like most want to get rid of the change address from BIP44. It also sounds like there are some other proposals on the table to try to come up with a new purpose (per BIP43) rather than blindly following BIP44 (since it makes less sense for Ethereum than it does Bitcoin)?
I am working on writing a simplified wrapper around the Ledger JavaScript libraries to make it easier to interact with them in Ethereum, but that requires deciding the structure of the path (so users of the library don't have to read through this thread and make a call on their own).
I'll update this as people respond to this thread so there is hopefully a source of truth for future readers. I think it is important that we all come to consensus on this as users with hardware wallets (e.g., Ledger/TREZOR) moving between products will expect those different products to all show them the same thing. At the moment, if I want my dApp to be able to use a hardware wallet managed primarily by Jaxx for one user and Electrum for another user I'll have to ask the user what "style" of wallet are they using (Jaxx vs Electrum) which results in a pretty poor UX.
Live Implementations:
m/44'/60'/0'/0/x: BIP44, MetaMask, Jaxx, MyEtherWallet (default), TREZOR App, Exodus
m/44'/60'/x'/0/0: BIP44, KeepKey, MetaMask (custom), Ledger Live
m/44'/60'/0'/x: Electrum, MyEtherWallet (ledger), Ledger Chrome App, imToken
m/44'/coin_type'/account'/0: Coinomi
Being forced to think about this whole situation more deeply, I am going to say that the only implementations (above) that are reasonable are m/44'/60'/0'/0/x. BIP 43 states that the first segment (44' in this case) is meant to define what the rest of the segments mean. BIP 44 then very explicitly states that for purpose 44', the segments mean m / purpose' / coin_type' / account' / change / address_index. If you don't like those segments that is perfectly fine and reasonable (per BIP 43 and BIP 32), but you MUST choose a new purpose (not 44'). This means that m/44'/60'/0'/x and similar are not reasonable because they are asserting that the rest of the path is BIP 44 compliant and then proceeding to not actually be BIP 44 compliant.
I am still a fan of coming up with a new purpose that makes more sense for Ethereum, but in the meantime I would like to encourage/pressure app developers to stop using the 44' purpose without following the BIP 44 spec. Pick any other number besides 44 and 0 for the purpose and then put whatever you want.
As an app developer, setting the purpose to 44 and then not following it with a BIP 44 path just makes everyone's life harder because we can't look at the path and determine what the intent was. It becomes a game of guessing.
Note: The usage of 60' above assumes Ethereum main network. Implementors SHOULD provide users a mechanism for indicating network (Test, ETH, ETC) and adjusting the path according to SLIP44.
Another live implementation, Exodus (m/44'/60'/0'/0/0) (see comment: #84 (comment)) - I'm willing to change it (if the path becomes standardized) so we can be compatible with all wallets, not just a few.
@MicahZoltu that's why I spent a bunch of time researching, and eventually making the decision to use the standard exactly as specified. Your solution seems good.. defining another purpose is fine.
We should absolutely specify a new purpose - iff we can convince all existing implementations to transition to it.
For my part, I'm happy to commit Ether Cards to transition, and I think geth will too, if sufficient others have committed. What do other implementors say?
@Arachnid MetaMask can transition for newly created accounts as necessary when the spec is specified
also curious about including network_id/chain_id in the derivation path so there is not info leakage across chain
If we're going to define a new purpose, there's no need to adhere to SLIP44 for chain ID - we should instead use the standard chain ID value as used in transaction signing.
The only input I have is that while I was the lead developer for Jaxx, I ran into an issue with testnet always being 1. I'd suggest a base code (60 for ethereum) plus a (very large) additive component (+10000) for testnet.
The problem occurred when trying to generalize, and offer wallets for testnet tokens that conformed to some type of standard that also would be not using the exact same path for ethereum testnet, btc testnet, and so on. Who needs to use this in production, I'm not sure, but it definitely caused a conflict in what the spec was supposed to represent and what ended up happening.
tldr; for the next purpose, try to keep HD testnet paths distinct per coin type.
@psionic81 Can you elaborate on the reason for this? With replay protection now in place, is there any reason to derive different keys for different networks, or different tokens?
Okay; I thought you were talking about my submission #600 / #601. I'm not sure where Bitcoin's testnet comes into it?
It's certainly not practical to store every ERC20 token in a separate wallet, because tokens need gas in order to be transferred, and the ether for that gas has to come from somewhere.
Have the majority agreed with #84 (comment) by @MicahZoltu ?
I've found that some wallet apps have stubbornly stuck with their non-standard implementations. @erasmospunk, please make Coinomi be BIP44 standard-compliant. I like Coinomi but cannot use it if its developers refuse to comply with standards.
@tab00 BIP32's receive/change addresses structure does not map to Ethereum. The first lightweight wallet to use BIP44 with Ethereum tried to emulate Bitcoin (was creating new addresses after one was used). Unfortunately this approach was broken, thus the alternative path was created to avoid the situation where restoring from seed would show an incomplete balance and transaction history.
@erasmospunk While following BIP44 may not make sense for Ethereum, we SHOULD follow BIP43 which states that if you start with m/44' then the rest of the path MUST be BIP44. If you feel like BIP44 doesn't make sense or is broken for Ethereum, that is fine, just use a prefix other than m/44'.
@MicahZoltu extended BIP44 defines only coin type indexes, not how each coin uses the account private key.
If you take Monero as an example, it is by definition incompatible with the proposed standard of BIP32 (different curve, the priv/pub is created with the transaction data, has spend & view private keys).
In case of Ethereum, many users are using the alternative paths and will be confused when missing their addresses.
BIP32 doesn't define any consistent path.
BIP43 asserts that the first segment of the path is a purpose, which defines what the remaining segments are. m / purpose'
BIP44 defines purpose 44 and sets the remaining fields to m / 44' / coin_type' / account' / change / address_index
SLIP44 further defines coin_type 60' to be Ethereum and 61' to be Ethereum Classic. m / 44' / 60' / account' / change / address_index
This leaves the last 3 fields as mildly ambiguous in Ethereum since Ethereum doesn't have a concept of a change address. If your first segment is 44' and you are writing an Ethereum wallet then the second segment MUST be 60'. From there you MUST have 3 more segments. If you do this, then at least you are conforming somewhat reasonably to BIP43, BIP44 and SLIP44 and I wouldn't fault you for choosing whether account' or address_index is what gets incremented, though it does feel like the only reasonable option for change is 0 (willing to hear debate on this last point if someone feels differently).
The problem I have with several of the implementations out there such as the ones used in (Coinomi, Electrum, MyEtherWallet (ledger), Ledger Chrome App, imToken) is that they use 44' as the purpose but then they proceed to not follow it with even the correct number of following segments as defined in BIP44. By using 44' as the first segment in the path you are asserting to readers that "this is a BIP44 path". By then not even having the right number of segments you are violating that assertion directly.
The other two implementations (m/44'/60'/0'/0/x m/44'/60'/x'/0/0), still differ, which is unfortunate, but they at least adhere as closely as possible to BIP44 when asserting the purpose of 44'. In both, it is just a matter of whether or not the indexing field is hardened or not, so both are legitimate uses of BIP44 paths and provide different security benefits.
Once again, my vote personally is that we come up with an Ethereum specific path (some have already been proposed) and we stop trying to follow BIP44. It is unclear to me why people thought it was a good idea to use BIP44 paths for Ethereum in the first place.
(You didn't @ mentioned me and I missed your comment)
@MicahZoltu BIP44 is for Bitcoin, you cannot apply it to Ethereum.
You didn't answer how you would handle Monero in the BIP44 context. If a coin has an EC curve different from secp256k1, you wouldn't do BIP32 public derivation unless you really wanted to lose money.
The change / address_index is bad for Ethereum for several reasons:
- Gives the impression that you can implement a wallet that don't reuse addresses. This has been attempted with very bad results.
- Gives the impression that the
change / address_indexaddresses are separate accounts. Those pseudo-accounts don't provide the BIP32 account isolation leading to unexpected security implications.
You are asking for an "Ethereum specific path" but the m/44'/60'/account'/0 -> account address was just that, a customised path for Ethereum.
BIP32, BIP43 and BIP44 are all designed to work with any coin, not just Bitcoin. Having Ethereum specs that needlessly collide with Bitcoin specs when the Bitcoin specs were designed with altcoins in mind just causes confusion and bugs (we already have problems with this collision in Ethereum as seen by the fractured tooling referenced in this EIP).
I'm all for a custom Ethereum path that doesn't use change field, and several have been proposed in other EIPs. I only argue that it shouldn't start it with m/44' because that is already well defined in BIP44 and there is no reason for us (Ethereum community) to intentionally be incompatible when the BIP authors took altcoins into account during their design process.
All of the confusion I have seen from users comes from the inconsistencies across tools. Most of the confusion I have seen from developers comes from them looking up the HD path specs online (which leads them to BIP32, BIP43 and BIP44) and not understanding why the accounts they are creating don't match what other tools create, followed by confusion why their paths match some tools.
I'm not sure why you are asking about Monero here, I'm not a Monero developer and know almost nothing about them (technically), so I cannot answer your questions about how it works.
When I say "Ethereum specific path" I mean something that doesn't collide with the currently well defined and specified BIP44. We have an effectively infinite space to work with, there is no reason to confuse people by colliding with BIP44.
@MicahZoltu the m/44'/60'/account'/0 path was created specifically to address the one address per account nature of Ethereum, while leaving space for extensions with m/44'/60'/account'/x for x > 0.
I argue that the m / 44' / 60' / account' / change / address_index is broken and semantically unfit for Ethereum.
In Monero you have a "spend" and a "view" keys that are needed to create an address and this doesn't map to the Bitcoin model as well. This doesn't mean that it cannot use the BIP44 while defining it's own standard if needed.
It is better to accept the de facto alternative paths and recommend new implementations supporting the sane m/44'/60'/account'/0 path that was created specifically for Ethereum.
One could easily use the XKCD comic to argue that what I'm suggesting is to not create a new standard. 😁 There already exists a perfectly fine standard BIP32 and BIP43. We should follow them, and BIP43 has a definition on how to extend it to new systems (like Ethereum).
It is better to accept the de facto alternative paths
This is what I have a problem with. As shown in #84 (comment), there is not a defacto standard on which to rally. There are 4 separate mechanisms used by a variety of different tools. Since there isn't consensus, we should standardize on something that doesn't directly collide with BIP43 and BIP44 rather than picking one of the 4 existing options that does collide.
If we want to go with one of the defacto standards rather than coming up with a new Ethereum specific BIP43 extension then we should pick a non-colliding one. If we want to come up with a new Ethereum specific BIP43 extension then it means we don't follow any of the current implementations.
@MicahZoltu when a user creates an account with a specific path it cannot change and using a new path means that they have to move their coins. There are millions of people using different paths, hence the "de facto standards".
@MicahZoltu Something to add to your "single point of truth" comment is that Ledger are switching to m/44'/60'/x'/0/0 in their new desktop app, Ledger Live.
I'll update this as people respond to this thread so there is hopefully a source of truth for future readers. I think it is important that we all come to consensus on this as users with hardware wallets (e.g., Ledger/TREZOR) moving between products will expect those different products to all show them the same thing. At the moment, if I want my dApp to be able to use a hardware wallet managed primarily by Jaxx for one user and Electrum for another user I'll have to ask the user what "style" of wallet are they using (Jaxx vs Electrum) which results in a pretty poor UX.
Live Implementations:
m/44'/60'/0'/0/x: BIP44, MetaMask, Jaxx, MyEtherWallet (default), TREZOR App, Exodus
m/44'/60'/x'/0/0: BIP44, KeepKey, MetaMask (custom), Ledger Live
m/44'/60'/0'/x: Electrum, MyEtherWallet (ledger), Ledger Chrome App, imToken
m/44'/coin_type'/account'/0: Coinomi
Correct me if wrong, but at least Ledger Live and Metamask have derivation paths m/44'/60'/x'/0 for their first address
@xardass Ledger Live is m/44'/60'/x'/0/0. MEW, unless it has changed, is m/44'/60'/0'/0/x.
Something to keep in mind is that some tools append the last segment from the index of a list of addresses, and then say that the derivation path is everything but the last segment. This is incredibly confusing for users because when you ask them, "what derivation path did you use" they tell you what the UI shows which is not correct.
Could one interim fix be for the wallet software to continuing trying other paths (in the background), and only if no balance is found on the first path, and then if no balance is found on any of the other paths it would automatically revert back to the first path (as the user may be initializing a brand new unused address) Would such a workaround be feasible?
Yeah, and I believe some wallets do this. It is not a great user experience overall but it is better than the user not being able to find their money!
Correct me if wrong, but at least Ledger Live and Metamask have derivation paths
m/44'/60'/x'/0for their first address
For the first address x=0, so both, Ledger and Metamask will use m/44'/60'/0'/0/0. But the following addresses will differ. For example, the second address for Ledger will be derived from m/44'/60'/1'/0/0 while the second address for Metamask will use m/44'/60'/0'/0/1.
How to send from multiple addresses to one.
So not 1 to 1 BUT multi to 1 ?!
There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.
This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.