DigiByte-Core/digibyte

importaddress shows date of import (date/time of scanning) instead o of real tx date

skironDotNet opened this issue · 12 comments

latest version (8-RC4) from develop with commit 7d39358
I use TrustWallet and they suck, they are not showing full history, and despite tons of ppl complaining and devs promising to fix, they still don't show full history.

I decided to take the public address (I do have private but...) and import to Digibyte Core as watch only, the importaddress command and it imported all the history but the date is the scanning date which for me is today, so this is bit useless history.
Can this be fixed?

Oh, I'm looking at the history and I noticed that also when I do private key import as importprivkey "privkey" ( "label" rescan ) and I put false for rescan on purpose to first import multiple priv keys, and then run command rescanblockchain ( start_height stop_height ) with providing start and stop (although may be not relevant) then the tx date is also scanning date not the actual tx date.

To me this is a serious bug even if 99% of users would not use the wallet such "advanced" way

P.S. as far as version I think this bug was present in older versions too, in fact many other coins/wallets have same bug

Thank you for taking the time to describe this issue. This does sound like a rather quirky issue.

To clarify this issue, you are first attempting to import multiple wallet public addresses without private keys before you do a re-scan? But after importing all the addresess and then doing a rescan you are saying a lot of tx data is missing?

This is to be expected if you don't have txindex=1 in your conf file.

If you are using rescanblockchain ( start_height stop_height ) then you have to make sure the block heights your referring to actually include the TX dates your looking for after you import private keys. Best options is a full rescan. But without txindex=1 and a reindex they likely wont show.

What flags do you have in the DGB config file? Also what type of private keys are you importing? Is it an HD wallet?

Do you have server & txindex enabled? On any wallet I use to scan the chain for addresses or interface for an app I always have txindex enabled and server. The re-index is needed once after you enable txindex unless you sync fresh with it on. Then it can be disabled.

reindex=1
txindex=1
server=1

By default, DigiByte Core builds a database containing only the transactions related to the user’s wallet (private keys). If you want to be able to access any transaction with commands like gettransaction, you need to configure Bitcoin Core to build a complete transaction index, which can be achieved with the txindex option. Set txindex=1 in the DigiByte Core configuration file

DigiByte Core doesn't keep a transaction index of transactions that are not specific to the addresses generated by your wallet private keys by default. By enabling txindex (txindex=1) you will make DigiByte Core build the transaction database index and index them for all other transactions to be found easily.

Typically using the watch only feature was to see real time movements, not a full tx history associated to address like you see with your own private key address. This is why block explorers were invented.

Have you tried looking the address history up on DigiExplorer.info? This service relies on a core wallet runningtxindex=1 and server=1 which then builds its own DB from the core wallet utxoset.

https://digiexplorer.info/

@skironDotNet Any update on this?

Sorry, missed during volume of work. I need to test with txindex=1 as Jared proposed and I will get back to you soon

AH! after hours of reindexing I forgot to remove the flag from the script that started the client, and started reindexing again :(
Will run it manually this time to avoid same mistake...

AH! after hours of reindexing I forgot to remove the flag from the script that started the client, and started reindexing again :( Will run it manually this time to avoid same mistake...

You are not the first person to have that happen to them! :)

Don't want to deviate the original issue, but I wonder if there could be a check for most recent reindex and warn before staring the operation again :)

I experienced a problem during testing. My private key has been generated using walletgenerator[.]net the priv key imported ok to v7.17.3 but UTXO not found there due to back N blocks bug that has been solved recently in yet unreleased version 8.x
I'm using my own build of that unreleased version and when trying to importprivkey getting error Invalid private key encoding (code -5) the format of the priv key is 6JXc...u213

I will get other priv key from another wallet but this is sad that far 2016 legacy priv key can't be imported, it ruins the concept of long term cold store or paper wallet

Going back to original issue. I confirm that having re-indexed blockchain with txindex=1 makes rescanblockchain work properly and it creates proper date time for transactions for watch only importaddress and owned address importprivkey previously used with the argument rescan=false

Still the question I have is why when txindex=0 it does not? It can detect amounts of addresses so it must be using "raw transaction"

...
 "vout": [
    {
      "value": 1234, <--------------Amount
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 ... OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "...",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "xxx"
        ]
      }
    },
    {
      "value": 56, <--------------Amount
      "n": 1,
      "scriptPubKey": {
        "asm": "0 ...",
        "hex": "...",
        "reqSigs": 1,
        "type": "witness_v0_keyhash",
        "addresses": [
          "..."
        ]
      }
    }
  ],
  "hex": "...",
  "blockhash": "...",
  "confirmations": 52030,
  "time": 1710649911, <-------------------------------- is this not available when txindex=0 ?
  "blocktime": 1710649911

Another discovery. I took the backup copy of the blockchain that was created without tx index (txindex=0) and repeated the steps above on brand new wallet file, and the date is also correct, so I'm confused now myself. Maybe the date was not correct when done importwallet after previously dumpwallet

Please give more time to, re-test everything, all the possible combinations...

I had a glitch in testing. After setting txindex=1 the folder indexes got created in data folder, so what it means in practice is that making txindex=0 still uses the indexes from that folder and the test was invalid to compare.
What I did after this discovery, I copied blokchain to another folder, in datadir A I deleted indexes folder and set txindex=0 and re-indexed in B I kept the setup with txindex=1.
Having 2 instances of QT running I did exactly same testing procedure of importing 1 pub address only and 1 priv key, then executed rescanblockchain start end, where start was set before pub address first tx, and end after priv key last tx.

For my surprise both instances scanned and address same transactions with same amounts and same VALID date.
So the only reason I can think of I reported this bug was maybe because of having blockchain downloaded by v7.17.3 and maybe lower versions in the past, and maybe switching to v8 required reindex so when using "old" format of the blockchain with new client, it didn't have all the info in correct format for v8 thus the resulting tx date became the date of the day when I scanned for transactions after importing pub/priv keys

I don't get it. I just Re-Created a wallet with a set of custom sethdseed, did the full rescanblockchain and every date of every transaction is from today, like date the of tx discovered not the actual date of when tx has happened.
Oh... maybe need to have unlocked wallet with walletpassphrase before scanning, because I re-created the wallet as encrypted wallet.

Final instructions to myself.

Make sure txindex=1 in digibyte.conf and the chain is indexed

Create new wallet:

  • Check mark Encrypt Wallet
  • Check mark Make Blank Wallet

Go to console,

  • use walletpassphrase to unlock the wallet and USE LONG unlock time because scan takes long
  • use sethdkey true privkey_here
  • use keypoolrefil X where X is known range of addresses used
  • use rescanblockchain to have a list with proper dates. You can use start_height arg, but if you won't go low enough to include very first tx, then the next time you do rescanblockchain even from 0 it will NOT have proper date. All tx must be scanned in the first shot with unlocked wallet to have proper dates.