Error when construct a public key using uncompressed hex

Question

Error when construct a public key using uncompressed hex

Closed this issue 4 years ago · 6 comments

Hi,

I found construction of public key using uncompressed hex always fails, for instance:
https://www.blockchain.com/btc/tx/89baa6934a2e790699804c1ae4a0b8f4e537244fb4dc78e4cdd5aabf59b2ea20

in output 0 of the above transaction, the pkscript is:

{'asm': '1 045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf2 04e761169a78113ad09af84c5189e924155fd9a48a5acc13952cbbd797bfa47c56fa5e0649f48d5b1536a1fb1866ae237d95735e38a30f08a96cb7682ea9592c99 02f884647516015a0f5861633406a0c8b2ef2d8967aa61a97a5fc2b14bf0ce23ba 3 OP_CHECKMULTISIG', 'hex': '5141045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf24104e761169a78113ad09af84c5189e924155fd9a48a5acc13952cbbd797bfa47c56fa5e0649f48d5b1536a1fb1866ae237d95735e38a30f08a96cb7682ea9592c992102f884647516015a0f5861633406a0c8b2ef2d8967aa61a97a5fc2b14bf0ce23ba53ae', 'reqSigs': 1, 'type': 'multisig', 'addresses': ['143ECVfiiA3wJVuqWK5z5e9r6pMA1TUnya', '18HCsjRGfaHqvW82Ba2UqDkiVZHR9UpSLk', '1FvXo7PSxHzFx79AwFF49zPaugGR7G5bb3']}

There are three public keys corresponds to three addresses. The first two are uncompressed, and the last one is compressed. When I try to transform the first uncompressed key to address,

s = "045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf2"
comp = False
if len(s)==66:
  comp = True
pub_key = PublicKey.from_hex(s)
address = pub_key.get_address(compressed=comp).to_string()

it reports an error: ecdsa.keys.MalformedPointError: Point does not lie on the curve

but it works well with the compressed key.

Is it a bug? Thanks.

Answer 1 · 2021-03-11T09:45:33.000Z

These are not valid uncompressed public keys.

The following is a valid public key that will work: "0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8"

Now, the reason that these fake public keys are in the blockchain is because in the past when people wanted to add data in the blockchain they were using multisig outputs (such as the above tx) to encode the data they wanted to store. These would never be spent, since they are not valid, but still the data would be stored in the blockchain for ever. There are other ways to store data and you can read more on that here:

http://www.righto.com/2014/02/ascii-bernanke-wikileaks-photographs.html

Now such storing is avoided and frown upon. Using OP_RETURN is the way to go to store data in the blockchain.

Answer 2 · 2021-03-15T02:03:35.000Z

These are not valid uncompressed public keys.

The following is a valid public key that will work: "0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8"

Now, the reason that these fake public keys are in the blockchain is because in the past when people wanted to add data in the blockchain they were using multisig outputs (such as the above tx) to encode the data they wanted to store. These would never be spent, since they are not valid, but still the data would be stored in the blockchain for ever. There are other ways to store data and you can read more on that here:

http://www.righto.com/2014/02/ascii-bernanke-wikileaks-photographs.html

Now such storing is avoided and frown upon. Using OP_RETURN is the way to go to store data in the blockchain.

Thanks for your quick reply. You are right. However, I still have some questions.

In the script_pub_key, there are public keys in the 'asm' which should correspond to the 'addresses' . In the above example, the compressed one (the third one) can be transformed correctly. The other two uncompressed one doesn't work, and there are still two corresponding addresses.(So I have doubts if the decode code has some bugs.)

Since my purpose is to track balance of some addresses, my question is as follows:

'Address' may not be the same as the public key, so I cannot trust 'address', I should decode public key from the public key items by myself?
The above output has been spent, which can be observed in the blockchian.info., and there are only one valid public key, so I can deduce the transaction happens related to the valid one address?

Thanks.

Answer 3 · 2021-03-15T11:10:45.000Z

Hi JunFang,

With "above example" you mean https://www.blockchain.com/btc/tx/89baa6934a2e790699804c1ae4a0b8f4e537244fb4dc78e4cdd5aabf59b2ea20 ?

These are P2MS addresses, the old multisig outputs, which do not have a corresponding address. In order to lock funds into those (or identify them for their balance) one have to know all the public keys and recreate the script_pub_key (Pkscript). These are not used anymore. For completeness the specific example has a 1-of-3 multisig scheme that requires only a single signature to spend the funds. Thus, it has 3... the 2 of them being data and the third one being a normal public key that can be used to spend it!

Nowadays multisig is wrapped in P2SH and thus they have addresses.

I hope I have answered your question (since I wasn't certain I understood it). If not, it would be helpful if you provide some code with the respective keys so that I can run it and understand better.

Answer 4 · 2021-03-16T00:24:55.000Z

Hi, thanks for your comments.

Back to the example:

{'asm': '1 045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf2 04e761169a78113ad09af84c5189e924155fd9a48a5acc13952cbbd797bfa47c56fa5e0649f48d5b1536a1fb1866ae237d95735e38a30f08a96cb7682ea9592c99 02f884647516015a0f5861633406a0c8b2ef2d8967aa61a97a5fc2b14bf0ce23ba 3 OP_CHECKMULTISIG', 'hex': '5141045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf24104e761169a78113ad09af84c5189e924155fd9a48a5acc13952cbbd797bfa47c56fa5e0649f48d5b1536a1fb1866ae237d95735e38a30f08a96cb7682ea9592c992102f884647516015a0f5861633406a0c8b2ef2d8967aa61a97a5fc2b14bf0ce23ba53ae', 'reqSigs': 1, 'type': 'multisig', 'addresses': ['143ECVfiiA3wJVuqWK5z5e9r6pMA1TUnya', '18HCsjRGfaHqvW82Ba2UqDkiVZHR9UpSLk', '1FvXo7PSxHzFx79AwFF49zPaugGR7G5bb3']}

The two uncompressed public key are reported invalid using your library "pub_key = PublicKey.from_hex(s)", however, it can be correctly transformed into the addresses using bitcoinlib (https://github.com/1200wd/bitcoinlib), and the addresses are the same as the 'addresses' items in the script_pub_key

from bitcoinlib.keys import *

try:
   k = HDKey("045002000078da4d534b6e1b310cbd0a914d378e0f905d5bb48081a245d36c0a64c3d1d0334a34d240a43cf02e07692f9793f45176826e8c91453ebe0f75f3faf2")
except:
    print("invalid public key.")
else:
    address = k.address()
    print("%s, Compressed:%s"%(address,k.compressed))

Answer 5 · 2021-03-16T10:50:48.000Z

Thanks JunFang,

I reopened the issue and I will look into it thoroughly.

Answer 6 · 2021-03-18T10:39:48.000Z

I checked the linked library and it does not check the public key for validity of the public key (on the Bitcoin-used elliptic curve). It uses it as just some data which is possible to convert to address or anything else.

https://github.com/1200wd/bitcoinlib/blob/7a18b60601a88dd1809cfed367abc00ea0d2997b/bitcoinlib/keys.py#L747

The error you get is quite to the point. The point of the input (public key) is outside the elliptic curve (i.e. is invalid). From the point of view of my library anything you try to do with this key would be wrong.

I guess, I could add an option to ignore validity but to be honest I believe it protect users better as is -- and it would involve considerable changes because of my assumption that only valid keys are accepted.