Cannot be derived from xpublic_key

Hi! How are you?
Thank you very much in advance for the work! It is an excellent library.

I report a problem:
You can only make derivations from root_xplublic_key and from root_xprivate_key. You get the error:
"PermissionError: You can't drive this master key."
But not from xpublic_key (BIP32):

Example: BIP32 Derivation Path: m / 44 '/ 0' / 0 '/ 0

as if you can see it on this site:
https://iancoleman.io/bip39/

Thank you very much again. Greetings

Hey @gerag14, If on your path have hardened keys, it does not work for root xpublic key, it only works for root xprivate key to drive. You can also refer #15 (comment) issue.

I understand.

But if the last derivation (using from_index) is unhardened. Shouldn't it allow deriving from xpub?

I even commented on checking if you are root in the hdwallet.py file at line 487 and it works.

It doesn't work because it starts with 44' key, it requires root xprivate key. It also reads one by one, Sir. Use this
m/44/0/0/0 instead of this m/44'/0'/0'/0 derivation. Thanks!

I shouldn't be able to bypass from an xpub if my LAST bypass was hardened. But in the path (m / 44 '/ 0' / 0 '/ 0) the last derivation is not enforced. Therefore (BIP32) should be able to derive from an xpub and the derivations would be:

m / 0/0
m / 0/1
m / 0/2

I attach the photo of the test.

Thank you in advance for all the contribution!

Thanks @gerag14, Attach it again please. let me see it

@gerag14 is correct. Unhardened key derivation works with only the xpub key (also related to issue #15) — that's the very point of derivation in this context.

As a side note, throwing a PermissionError in situations like this is completely incorrect since the meaning of this exception is related to OS-level errors (it inherits from OSError), not general purpose "you can't do it" errors.

@meherett It looks like removing lines

python-hdwallet/hdwallet/hdwallet.py

Lines 487 to 488 in cd8c4f5

    
           if not self._root_private_key and not self._root_public_key: 
        
               raise PermissionError("You can't drive this master key.")

is sufficient for this to work.

Hey @alecov, I think you are using this from_xpublic_key function. Derivation is not working for from_xpublic_key function. Use this from_root_xpublic_key instead of from_xpublic_key function. Just see both examples for Root XPublic Key derivations and XPublic Key https://github.com/meherett/python-hdwallet/blob/master/examples/from_root_xpublic_key.py and https://github.com/meherett/python-hdwallet/blob/master/examples/from_xpublic_key.py.

For PermissionError, I will change it into ValueError exception.

Thanks.

Hi @meherett. I think the issue here is still not clear to you, so let me try to explain it better.

When deriving from an unhardened path (such as m/0 or m/1/2/3/4), you do not need to have the corresponding private key. You only need the public key part. This is in accordance to BIP-32's derivation rules, quote:

The function CKDpub((Kpar, cpar), i) → (Ki, ci) computes a child extended public key from the parent extended public key. It is only defined for non-hardened child keys.

That is, the derivation function exists from xpub to xpub, no need for the corresponding parent xprv. Since the xpub encodes its exact node inside the hierarchy, it also encodes the i index above (the depth) and other details needed for derivation.

This is furtherly elaborated here:

N(m/a/b/c) = N(m/a/b)/c = N(m/a)/b/c = N(m)/a/b/c = M/a/b/c.
N(m/aH/b/c) = N(m/aH/b)/c = N(m/aH)/b/c.

Note how you can derive the xpub from eg. m/a/b, which is N(m/a/b), from N(m/a) alone.

As for your suggestion to use from_root_xpublic_key(), deriving from an arbitrary xpub with from_root_xpublic_key() does not work because this function resets the internal key structure info to be that of the root key (depth = 0, parent fingerprint = 0, etc.), instead of using the encoded material from the key. This is clearly visible from:

wallet = hdwallet.HDWallet().from_root_xpublic_key("any-non-root-xpub", strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() # Fails

All of this means the following must work:

wallet1 = hdwallet.HDWallet()
wallet1.from_root_xprivate_key("any-valid-xprv")
wallet1.from_path("m/1'/2'/3'") # *Any* path
xpub = wallet1.xpublic_key()

wallet2 = hdwallet.HDWallet()
wallet2.from_xpublic_key(xpub)
# The following assert must pass:
assert wallet1.xpublic_key() == wallet2.xpublic_key()

The good news is that your code is already correct in all of the above regards, excluding the forementioned check which I suggest you to alter to perhaps:

if not self._public_key \
    and not self._chain_code \
    and not self._depth \
    and not self._index \
    and not self._parent_fingerprint:
    raise ValueError("Derivation not possible without the public key.")

This allows hdwallet to create watch-only addresses derived from arbitrary xpubs.

Please let me know if the issue is clearer to you now. I can also submit you a PR with the above.

Hey @alecov again, Yes the issue is cleared, sir.

NOTE: In v1 of HDWallet the from_xpublic_key() function is not drivable but instead of this the from_root_xpublic_key() function is drivable.

In this code without drive, the test does not fail.

wallet = hdwallet.HDWallet().from_root_xpublic_key("any-non-root-xpub", strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() # Not Fails

It fails in here.

xpublic_key = "any-non-root-xpub"
wallet = hdwallet.HDWallet().from_root_xpublic_key(xpublic_key, strict=False)
assert wallet.xpublic_key() == wallet.root_xpublic_key() == xpublic_key # Fails

Because the information of your Non-Root XPublic Key changed into Root XPublic Key. Both (Non-Root XPublic Key and Root XPublic Key) have the same Public Key, Chain Code. The changes are exhibit on depth, parent_fingerprint and index values to 0.

https://github.com/meherett/python-hdwallet/blob/master/hdwallet/hdwallet.py#L279

But, It does not affect driving XPublic Keys. So, I already fixed this issue by adding extra option change_to_root=False param on from_root_xpublic_key() function.

In the upcoming version-2.0.0 of HDWallet, I will consider merging up from_root_xpublic_key() and from_xpublic_key() functions. This is because lots of comments on this issue were raised by users of this HDWallet library.

Version-2.0.0 will come up with Comand Line Interface (CLI) and I will release it in these 2 weeks.

can you guys explain something. I tried work with private key and i receive same error:

hdwallet = HDWallet(symbol=BTC)
xprev_key="xprv9z7iGvkix69hxkUo44zia4SM8omSDsVgiftMpfFMmnQT4ThsBy4AgfRLgSuygTHsghwoUAPxqY1JyACycDVd2KdXvCDsuoAPch9UuiCXsoq"
hdwallet.from_xprivate_key(
xprivate_key=xprev_key)

    # hdwallet.clean_derivation()
    hdwallet.from_path(path="m/44'/60'/0'/0/0")

===========>>>>>>>>>>>>>>>>>>error
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/tests/test_hdWalletProvider.py", line 60, in test_hdwallet
hdwallet.from_path(path="m/44'/60'/0'/0/0")
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/venv/lib/python3.8/site-packages/hdwallet/hdwallet.py", line 445, in from_path
self._derive_key_by_index(int(index[:-1]) + BIP32KEY_HARDEN)
File "/Users/dmitryshlymovich/workspace/PycharmProjects/test1/venv/lib/python3.8/site-packages/hdwallet/hdwallet.py", line 488, in _derive_key_by_index
raise PermissionError("You can't drive this master key.")
PermissionError: You can't drive this master key.

as i understand it's should work with hardening with xliv? i miss something?

Hey @shdmitry2000, For derivation, use this from_root_xprivate_key() function instead of from_xprivate_key() function.

NOTE: In v1 of HDWallet both from_xpublic_key() and from_xprivate_key() functions are not drivable but instead of this both from_root_xpublic_key() and from_root_xprivate_key() functions are drivable.

So, For your issue, check this one

XPRIVATE_KEY = "xprv9z7iGvkix69hxkUo44zia4SM8omSDsVgiftMpfFMmnQT4ThsBy4AgfRLgSuygTHsghwoUAPxqY1JyACycDVd2KdXvCDsuoAPch9UuiCXsoq"
hdwallet = HDWallet(symbol="BTC")
hdwallet.from_root_xprivate_key(xprivate_key=XPRIVATE_KEY, strict=False)
hdwallet.from_path(path="m/44'/60'/0'/0/0")
print(hdwallet.p2pkh_address())

Thanks!

Hi @meherett,

In the upcoming version-2.0.0 of HDWallet, I will consider merging up from_root_xpublic_key() and from_xpublic_key() functions. This is because lots of comments on this issue were raised by users of this HDWallet library.

This makes sense to me as well. There's little point in separating root- from non-root keys in a library, specially because xpubs/xprvs are self-contained (they contain everything needed for derivation).

Thanks for the good work.

Hey @alecov, It's fixed on v2.0.0 package. https://github.com/meherett/python-hdwallet/releases/tag/v2.0.0

Thanks!

v2.0.0 bug issues
please check again the error file tests/hdwallet/test_from_xpublic_key.py

root_xpublic_key

It doesn't make sense for address generation tests/values.json

root_xpublic_key modify to bitcoin_account_extended_public_key
bitcoin_account_extended_public_key:"xpub6CnRmcvtuVkmYV7UrPbrrhpJaor6iy5cFLQHdsgYQsfz9338T6Wqy3vHP1MXCEPMKzq1C8fErhrGnzEYarzsF814HnCzNF6vqo9JiGjjvE2"

bitcoin_account_extended_public_key path m/44'/0'/0'

bitcoin_account_extended_public_key Must be obtained through BIP32 Root Key.

My test environment configuration

from bitcoin.mainnet.mnemonic in tests/values.json

key	value
BIP39 Mnemonic	avocado harsh input rack like desert mask nominee left atom original tower shy barely base
BIP32 Root Key	xprv9s21ZrQH143K3nQRmmivfV9RstNXkiuGcp6XCaUQyNeD4Zczidb8K3ue3LCVSA47aRqvbpVj5NKLxv8DJi1adCzPmzXVJiyJ7o5WkRTDuFe
BIP44 Path	m/44'/0'/0'/0/0
BIP44 Path p2pkh	16W7JeQXEzmYKX8UMFMrg6FwNSPduVdYHo
BIP44 Account Extended Public Key	xpub6CnRmcvtuVkmYV7UrPbrrhpJaor6iy5cFLQHdsgYQsfz9338T6Wqy3vHP1MXCEPMKzq1C8fErhrGnzEYarzsF814HnCzNF6vqo9JiGjjvE2
BIP44 Account Extended Public Key Path	m/44'/0'/0'

use BIP44 Account Extended Public Key

    hdwallet.from_xpublic_key(
        xpublic_key="xpub6CnRmcvtuVkmYV7UrPbrrhpJaor6iy5cFLQHdsgYQsfz9338T6Wqy3vHP1MXCEPMKzq1C8fErhrGnzEYarzsF814HnCzNF6vqo9JiGjjvE2"#_["bitcoin"]["mainnet"]["root_xpublic_key"]
        , strict=False
    )
    hdwallet.from_path(
        path="m/0/0"
    )
    assert hdwallet.p2pkh_address() == "16W7JeQXEzmYKX8UMFMrg6FwNSPduVdYHo"

iancoleman Web site for authentication

Hello Guys,

most HD wallet generate wallet like this using the root key:
m/44'/0'/0'/0/0
m/44'/0'/1'/0/0
m/44'/0'/2'/0/0

so there is a need to use the root public key to generate public address in that format, how do we go about this using this library.

kind regards

	if not self._root_private_key and not self._root_public_key:
	raise PermissionError("You can't drive this master key.")