How to manage fees for testnet?

[ruben2k1]

I don't understand why I'm getting this error if I'm applying the correct sats/byte (according to mempool, it's 1 sat/byte for testnet):

Error: Warning: you are paying about 0.05152416 in fees, which is 26976 satoshi per byte for a transaction with a VSize of 191 bytes (segwit counted as 0.25 byte per byte). Use the setMaximumFeeRate method to increase your threshold, or pass true to the first arg of extractTransaction.

I am checking this URL for utxos: https://mempool.space/testnet/api/address/mteJoFQgVGBbwkTsLT55Lv9ajwZkcoanJn/txs/chain

I would appreciate if someone could explain to me when is the difference between the txid of the object and the txid inside vin[]. I am confused and what I do is to add the txid of the object to the hash inside addInput() and also to query the transaction in hexadexacimal and pass it as Buffer to nonWitnessUtxo, I don't know if this is well done or I should add the txid of vin[] and its transaction in hexadecimal

The code is:

const txsInfo = await getAddressConfirmedTransactions(test1Address, bitcoin.networks.testnet);
const firstTx = txsInfo[0];
const transactionHex = await getTransactionHex(firstTx.txid)
    
psbt.addInput({
   hash: firstTx.txid,
   index: 0,
   nonWitnessUtxo: Buffer.from(transactionHex, 'hex')
});
    
psbt.addOutput({
   address: 'mjVRXqyc5SjECeGJy3FJL7d5J4YJjvuSzK',
   value: 25000
});
    
psbt.signInput(0, test1);
psbt.validateSignaturesOfInput(0, validator, buffPubKeyHex);
psbt.finalizeAllInputs();
psbt.setMaximumFeeRate(1);
    
const rawTx = psbt.extractTransaction().toHex();

console.log(rawTx);

const txId = await axios('https://mempool.space/testnet/api/tx', {
   method: 'POST',
   data: rawTx
})

console.log(txId.data);

If I change:
const rawTx = psbt.extractTransaction().toHex();
for this
const rawTx = psbt.extractTransaction(true).toHex();
I get back: data: 'sendrawtransaction RPC error: {"code":-25,"message":"Fee exceeds maximum configured by user (e.g.
-maxtxfee, maxfeerate)"}'

[junderw]

You are misunderstanding how fees work in Bitcoin.

1. setMaximumFeeRate will set "the ceiling fee rate above which the extractTransaction method will throw an error." so setting it to 1 means "if the resulting transaction has a satoshi per byte fee rate of over 1 sat/byte, then it will throw an error" (The default is currently 2500). You seem to think that this method will set the fee of your transaction. That is incorrect.
2. extractTransaction(true) says "I will ignore the absurdly high fee check"... so the JS will no longer throw an error. Mixing this with setMaximumFeeRate(1) doesn't make sense. You are saying "I want to lower the ceiling of the high fee check, but also NOT perform the high fee check at all"
3. Fees in Bitcoin are controlled by the total value of the inputs minus the total value of the outputs. So in order to control the fee, you should count the total value of the inputs into let totalInputValues = 0; and add up the values totalInputValues += firstTx.value; (I don't think the API you currently use has this info. use the utxos endpoint instead). Then you should add a 2nd output sending totalInputValues - 25000 - yourFees back to test1Address... Since it says your transaction is 191 bytes in the error, you should probably pay 200 satoshis. so yourFees should be 200. In your example transaction, 5177467 - 25000 - 200 would be 5152267 so you should add psbt.addOutput({ address: test1Address, value: 5152267 }); (And don't forget to delete the setMaximumFeeRate call.)

[junderw]

Re: the sendrawtransaction RPC error from Bitcoin Core.

Bitcoin Core has something similar to setMaximumFeeRate where they won't let you broadcast a tx with a high fee rate or a high fee.

You can change those values in your bitcoin conf file or with the runtime parameters of bitcoind, but that's not what you want to do here, you should modify the fee to make the fee itself lower.

[junderw]

Also let me add: You are doing a great job by testing out your wallet software on testnet.

I know many people who have lost actual money from bugs like this while they test their wallet... which is why we have this error in the first place.

[ruben2k1]

First of all, thank you for answering Mr. Underwood

I have changed my code. So, would this be okay?

I tried 200 but I got this API error: data: 'sendrawtransaction RPC error: {"code":-26, "message": "min relay fee not met, 200 < 225"}'

What does this mean? Is the API that does not accept less than 225 sats?

So I had to put 300 (the first approximate number I could think of)

Out of curiosity, do you know of any guide where I can learn how Bitcoin technically works

    let totalInputValues = 0;
    const txsInfo = await getAddressUtxos(test1Address, bitcoin.networks.testnet);
    const firstTx = txsInfo[0];
    const transactionHex = await getTransactionHex(firstTx.txid)
    
    console.log(firstTx);
    
    totalInputValues += firstTx.value;

    psbt.addInput({
        hash: firstTx.txid,
        index: 0,
        nonWitnessUtxo: Buffer.from(transactionHex, 'hex')
    });
    
    psbt.addOutput({
        address: 'mjVRXqyc5SjECeGJy3FJL7d5J4YJjvuSzK',
        value: 25000
    });
    
    psbt.addOutput({
        address: test1Address,
        value: totalInputValues - 25000 - 300
    });

    psbt.signInput(0, test1);
    psbt.validateSignaturesOfInput(0, validator, buffPubKeyHex);
    psbt.finalizeAllInputs();
    
    const rawTx = psbt.extractTransaction().toHex();

    console.log(rawTx);

    const txId = await axios('https://mempool.space/testnet/api/tx', {
        method: 'POST',
        data: rawTx
    })

    console.log(txId.data);

I also just sent another transaction after the previous one and it worked before with the same pubKeyHex (I have replaced some numbers with X for privacy):

Error: Can not sign for this input with the key 03d32ec6355078109f7519f762f658b020681914d04b0d29715345a3177XXXXXXX

[junderw]

1. min relay fee is a setting in Bitcoin Core (Default is 0.00001 BTC per kilobyte, aka 1 satoshi per byte) that decides the smallest fee rate to allow for propagating to peers. Your transaction was 225 bytes in size, so anything below 225 satoshis in fees will make the fee rate lower than 1 satoshi per byte and can not be propagated. When you set it to 300, it was 1.33 satoshi per byte (0.00001333 BTC per kB) so you were OK.
2. https://btcinformation.org/en/developer-guide is a good resource for developers.
3. Most likely the index value you passed into addInput is incorrect. This is not the index of your input that you are creating. This is the index of the output you are spending. So the Psbt code will look inside the nonWitnessUtxo transaction and find the index-th output and use that as the script you must "unlock"... if you pick the wrong one it sees that the pubkey hash of the output does not match your key and throws that error.

[ruben2k1]

I see, so the index is the corresponding vout (firstTx.vout)

So, how could I calculate the fee measured in sats/byte to know what I have to pay in commission before approving the transaction?

I mean to calculate the 225 fee sats but in a dynamic way

I have made a GET query to the URL: https://mempool.space/testnet/api/v1/fees/recommended and I get back:

{
   "fastestFee": 1,
   "halfHourFee": 1,
   "hourFee": 1,
   "economyFee": 1,
   "minimumFee": 1
}

[junderw]

That API returns the recommended fee rate for various levels of ability to wait.

The unit used is satoshi per vByte. vByte is essentially equal to bytes if you use pre-segwit scripts (like the one you're using) but with segwit, witness bytes are discounted at 75%... so 4 witness bytes equals 1 normal byte...

To account for this difference in meaning, we call it vBytes, vSize, virtual size etc... but just know that in your case, "vByte" = "byte"

Since you have satoshis/vByte (from the API), and you want to know satoshis... you must figure out vBytes (transaction size) and calculate satoshis with feeRate * vSize.

Since most script types have a known, set size... you can estimate the size of the transaction just by knowing the types of inputs and outputs and how many there are.

Here's a very rough calculation: https://gist.github.com/junderw/b43af3253ea5865ed52cb51c200ac19c

[ruben2k1]

Excellent, so the rate for my use case would be about 231 sats (1 sats/byte * 231 bytes)

I had read this issue and got confused: https://github.com/bitcoinjs/bitcoinjs-lib/issues/1566. Also with psbt.getFee() and psbt.getFeeRate()

I have found two methods in the Psbt class that supposedly returns the bytes of the transaction. Is this true and could it be used to then multiply by the sats/bytes?

psbt.extractTransaction().byteLength()
psbt.extractTransaction().virtualSize()

Both they return the same (225 bytes)

I have modified the return by adding 20% more sats

return Math.ceil(totalWeight / 4 * 1.2);

const bytes = getBytes({'P2PKH':1},{'P2PKH':1});
console.log(bytes);

[junderw]

const bytes = getBytes({'P2PKH':1},{'P2PKH':1});
console.log(bytes);

You forgot the change output sending back to yourself. You need 2 outputs, not 1.

* 1.2 shouldn't be necessary, but adding a little extra is fine.

[ruben2k1]

Oh thanks, I forgot it

Then I can use these methods to get the bytes or has nothing to do with?

psbt.extractTransaction().byteLength()
psbt.extractTransaction().virtualSize()

[junderw]

extractTransaction will give you the fully signed final transaction.

virtualSize will give you the vSize of that transaction. byteLength will give you the size of that transaction. (These will be equal for non-segwit transactions, but you will want to use virtualSize for segwit... so it's better to always use virtualSize now.

However, you need to know the vSize estimate in order to create the change output to yourself, and you need to know the change output and amount in order to sign, and you need to sign and finalize before calling extractTransaction... so it's a catch 22...

1. Use some sort of vSize estimation algorithm to guess the size, calculate the fee, and calculate the change output amount.
2. OR you could use a dummy change amount, sign, finalize, extract, get the actual vSize, then throw away the signed PSBT, then use the real vSize to calculate the real change amount and then re-sign, re-finalize, re-extract etc.

These are two ways to figure out change amount and adjust fees.