Utxo's

[Harvey44]

despite having unspent funds, confirmed from service providers

i still get error saying i don't have enough unspent output for transaction.

What could i be doing wrongly.

[mccwdev]

It could be a setting requiring a minimal number of confirmations.
Also you could check the logs if you see an error or warning.
If that doesn't solve the problem please post the code which results in this problem.

[Harvey44]

What i mean is, if balances are spread across different addresses in a wallet. When you want to make a transaction, you get an unavailable unspent or unavailable key error. You have to then use transactions update methods which works sometimes. Then the wallet outputs different balance each time you check with balance_from_serviceprovider method , you have to check all addresses with a different balance checker to really know the total wallet balance. But what's really going on there and what's the best solution. Why are different balances being given, and why can't you make a transaction despite having balance across different addresses in the wallet. Thanks for your time.

…

On Mon, Sep 6, 2021, 8:08 AM Lennart Jongeneel ***@***.***> wrote: It could be a setting requiring a minimal number of confirmations. Also you could check the logs if you see an error or warning. If that doesn't solve the problem please post the code which results in this problem. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#186 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AORKAAID2MVE7JAUIAOKZZTUARSGXANCNFSM5DPEPCNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[mccwdev]

Sounds like one of the service providers is returning wrong results, I will run some checks to see if I can reproduce anything.

If you have this issue again, please check the logs which providers are used and returns incorrect results.

Are you using the lastest version 0.6 of the library? Some provider issues are fixed in the latest version.

[Harvey44]

I updated to the latest version nw, and tried it but same error. I tried using the transactions_update method. I called the wallet asJSon() and the info() methods. The balance were the same, but am still getting No unspent transaction outputs found or no key available for UTXO's in the logs.

…

On Wed, Sep 29, 2021, 9:06 PM Lennart Jongeneel ***@***.***> wrote: Sounds like one of the service providers is returning wrong results, I will run some checks to see if I can reproduce anything. If you have this issue again, please check the logs which providers are used and returns incorrect results. Are you using the lastest version 0.6 of the library? Some provider issues are fixed in the latest version. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#186 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AORKAANNRCCU6TP4K3Q4VNLUENWUPANCNFSM5DPEPCNA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[dgtlmoon]

Hi, thanks for the amazing project!
version bitcoinlib == 0.6.3.

I can report something similar, when I used .send_to based on the wallet's total balance, I got the "Not enough unspent transaction outputs found" error, but if I iterated over each key available, and just sent what was available on that individual key, things worked fine

  # a .send_to(amount=int(wallet_balance-forward_fee),) based on this amount didnt work
  wallet_balance = wallet.balance(network=os.getenv("BTC_NETWORK", "testnet"))

but this worked

    forward_fee=1024

    for key in wallet.keys(network=os.getenv("BTC_NETWORK", "testnet")):
        if key.balance:
            t = wallet.send_to(to_address=os.getenv("BTC_FORWARD_ADDRESS", "xxxx"),
                               network=os.getenv("BTC_NETWORK", "testnet"),
                               amount=int(key.balance-forward_fee),
                               fee=forward_fee)

First i'm updating keys based on another table of keys that my application is interested in

wallet.utxos_update(networks=os.getenv("BTC_NETWORK", "testnet"), key_id=key.id)

HOWEVER.. when I run this again.. about 10 minutes later, I see the error again, so there's some state-issue where the .keys() doesnt have an option like confirms=1, or.. unsure.. basically i've spent the output, but it's still showing, however that output could be unconfirmed at this time

    for key in wallet.keys(network=os.getenv("BTC_NETWORK", "testnet")):
        if key.balance:
            # forward any new unspent inputs out somewhere, dont keep them on the server
            # seems to be more reliable than send of total wallet
            t = wallet.send_to(to_address=os.getenv("BTC_FORWARD_ADDRESS", "xxx"),
                               network=os.getenv("BTC_NETWORK", "testnet"),
                               amount=int(key.balance-forward_fee),
                               fee=forward_fee)

[dgtlmoon]

Ahhhh, looks like the key has double the balance somehow!

https://www.blockchain.com/btc-testnet/address/mowLeYZWK8S9CMcCWibPd2QqeWt6bDgcuT "0.00026409 BTC" total received

But I see key.balance = 52818

0.00026409 * 2= 52818

sqlite> select * from transaction_inputs where key_id=608;
sqlite> 
sqlite> select address,spent from transaction_outputs where key_id=608;
mowLeYZWK8S9CMcCWibPd2QqeWt6bDgcuT|0
mowLeYZWK8S9CMcCWibPd2QqeWt6bDgcuT|0

I tried wallet.transactions_update(network=os.getenv("BTC_NETWORK", "testnet"), key_id=608) but it still has the wrong info, I've tried the usual .scan() too

  b=wallet.balance_update_from_serviceprovider(network=os.getenv("BTC_NETWORK", "testnet"))

also returns double

So I send_to() the correct amount, it works the first time (as i would expect), but it lets me .send_to() again but without error, for the same amount (and obviously I see no 'new' amount showing in my testbtc client)

[dgtlmoon]

Attached is my wallet.db file :) shouldnt be anything in this one, and all balances should be returned to my testnet client

wallet.db.zip

[mccwdev]

It seems the wallet was not able to send a correct transaction, or the transaction was not accepted by the network so it remains unconfirmed in the wallet. It's hard for me to see how this transaction was created and how to avoid it.

You can delete the transaction, which solves the problem:

t = w.transaction('fd5d2655cb0dc6222560b85cefbf60d35410aa6bf08689a711928b842dc3d32d')
t.delete()

Please let me know if run into this issue again, and maybe you can reconstruct how the 'faulty' transaction was created.

[dgtlmoon]

@mccwdev thanks for the reply! I hit the issue again, it looks like this is the pattern

• make an address btc_key = wallet.new_key(network=os.getenv("BTC_NETWORK", "testnet"), name=btc_keyname)
• send to that address (I'm using Electrum on the testnet)
• immediately just start a loop of attempting to .send_to(..) out of the wallet when the transactions are unconfirmed (according to Electrum)

So it's first letting me spend based on a balance that is not confirmed, once the key is in this state, it seems to always be stuck there..

Any ideas for more debug? I'll see if I can really narrow it down further with a script

[dgtlmoon]

Hmm i dont know exactly, but it seems the balance_update functions are always saying min_confirms=0

bitcoinlib/bitcoinlib/wallets.py

Line 2864 in 0de3b77

self._balance_update(account_id=account_id, network=network, key_id=key_id, min_confirms=0)

bitcoinlib/bitcoinlib/wallets.py

Line 3107 in 0de3b77

self._balance_update(account_id=account_id, network=network, key_id=key_id)

even balance() hmm

bitcoinlib/bitcoinlib/wallets.py

Line 2587 in 0de3b77

self._balance_update(account_id, network)

[dgtlmoon]

I just tried again by sending a transaction, then waiting until there was 2~ confirmations before using .send_to and i can see the issue again

[dgtlmoon]

One clue..

print (key.transaction_outputs)

Traceback (most recent call last):
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "./app.py", line 270, in sync_unpaid_accounts
    print (key.transaction_outputs)
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 481, in __get__
    return self.impl.get(state, dict_)
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 941, in get
    value = self._fire_loader_callables(state, key, passive)
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 977, in _fire_loader_callables
    return self.callable_(state, passive)
  File "/home/dgtlmoon/.local/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 861, in _load_for_state
    raise orm_exc.DetachedInstanceError(
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <DbKey at 0x7f61f4374910> is not bound to a Session; lazy load operation of attribute 'transaction_outputs' cannot proceed (Background on this error at: https://sqlalche.me/e/14/bhk3)

[dgtlmoon]

Ok, here I can reproduce atleast the first part of the bug, in this simple program/script wallet.balance will immediately report a positive balance on unconfirmed transactions, there's no way to tell wallet.transactions_update to only include confirmed transactions, (which may or may not solve the overall wallet.balance being based on unconfirmed transactions)

interestingly, i

#!/usr/bin/python3

from bitcoinlib.wallets import wallet_create_or_open

if __name__ == '__main__':
    forward_fee = 1024

    wallet = wallet_create_or_open(name="test-bug", network="testnet")

    btc_key = wallet.new_key(network="testnet", name="my buggish address")

    input("Please send some amount to {} and hit enter".format(btc_key.address))

    print ("Updating transactions")
    # Update each key
    for key in wallet.keys(network="testnet"):
        wallet.transactions_update(network="testnet", key_id=key.id)
    print ("Wallet balance is {}".format(wallet.balance()))
    print("Returning/spending outputs")
    for key in wallet.keys(network="testnet"):
        if key.balance:
            print ("Key balance is {}, sending".format(key.balance))
            # Send it back to my wallet
            t = wallet.send_to(to_address="tb1q4fzu384ffu70js67h7vvcq837pvqe97kgv306q",
                               network="testnet",
                               amount=int(key.balance - forward_fee),
                               fee=forward_fee)

Here I see 50,000 satoshi's (correct for what I sent), the transaction is in 'unconfirmed' state according to Electrum,

Updating transactions
Wallet balance is 50000.0
Returning/spending outputs
Key balance is 50000, sending
....
    raise WalletError("Create transaction: No unspent transaction outputs found or no key available for UTXO's")
bitcoinlib.wallets.WalletError: Create transaction: No unspent transaction outputs found or no key available for UTXO's

I wait until Electrum reports 1 confirmation, then re-run the script, everything works fine, amount was correctly sent, balance set back to zero

I'm unable to reproduce the 'double amount' issue, just yet

[hammersharkfish]

@mccwdev Ok , I think I finally found the issue .

        if not selected_utxos:
            lessers = utxo_query. \
                filter(DbTransactionOutput.spent.is_(False), DbTransactionOutput.value < amount).\
                order_by(DbTransactionOutput.value.desc()).all()
            total_amount = 0
            selected_utxos = []
            for utxo in lessers[:max_utxos]:
                if total_amount < amount:
                    selected_utxos.append(utxo)
                    total_amount += utxo.value
            if total_amount < amount:
                return []

utxo.value is returning incorrect total . These were my unspent outputs at that time -https://www.blockchain.com/btc/tx/3578979689fcc272a54cecc1e8c65efa021a2a77fb6ca8eed2869f60df8224f5
utxo.value had value of 5260 . It should have had value of 6258(5260 +998)
Manually entering total_amount=6258 right after the for loop made the transaction go through . Now I am looking for a permenant solution .

[hammersharkfish]

Manually entering total amount frequently runs into bitcoinlib.wallets.WalletError: Fee per kB of -7635 is lower then minimal network fee of 1000 .

[doncoriolan]

I think I am experiencing the same issue. After creating a transaction with that still isn't confirmed. I am attempting to create another transaction and I get an error.

First transaction

>>> tx = mike.send([('tb1qffcct7hdhkvv3mhyg8jye5rf37gyctckqepd0c', 100000)], offline=False, network='testnet', fee='high')
>>> tx.info()
Transaction fbc8c94d69a5d86888f26dea52368be43a7637644ab98f1bb0a4e11b82577a69
Date: None
Network: testnet
Version: 1
Witness type: legacy
Status: unconfirmed
Verified: True
Inputs
- msdySyxSVBFKaoSPyVHNCWV32KoSPzLqu6 0.00050000 tBTC b85d1368c4e8c118543169737ca870bc15ac877535fe2b0efcf9d3782e66aad1 0
  legacy sig_pubkey; sigs: 1 (1-of-1) valid
- msdySyxSVBFKaoSPyVHNCWV32KoSPzLqu6 0.00099312 tBTC 23b916a68bc6bbe86b57a183598be25442cbde478ce4cbfb5c9448aa56f80dc1 1
  legacy sig_pubkey; sigs: 1 (1-of-1) valid
Outputs
- tb1qffcct7hdhkvv3mhyg8jye5rf37gyctckqepd0c 0.00100000 tBTC p2wpkh U
- msc3WkNa7GXXJRPsJBV7jGPpj8gd17moFj 0.00049112 tBTC p2pkh U
Size: 369
Vsize: 369
Fee: 200
Confirmations: 0
Block: None
Pushed to network: True
Wallet: mike

Second Transaction

>>> tx2 = mike.transaction_create([('tb1qffcct7hdhkvv3mhyg8jye5rf37gyctckqepd0c', 100000)])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/bitcoinlib/wallets.py", line 3528, in transaction_create
    selected_utxos = self.select_inputs(amount_total_output + fee_estimate, transaction.network.dust_amount,
  File "/usr/local/lib/python3.8/dist-packages/bitcoinlib/wallets.py", line 3381, in select_inputs
    raise WalletError("Create transaction: No unspent transaction outputs found or no key available for UTXO's")
bitcoinlib.wallets.WalletError: Create transaction: No unspent transaction outputs found or no key available for UTXO's

[crazypon]

bitcoinlib.wallets.WalletError: Not enough unspent transaction outputs found

occurs

config = configparser.ConfigParser()
config.read("bot.ini")
wallet_info = wallet.as_dict()
main_wallet = config["payments"]["wallet_address"]
# getting all balance in str
amount = wallet_info["main_balance_str"]
wallet.utxos_update(networks="testnet")
wallet.get_keys()
wallet.scan()
wallet.info()
transaction_hash = wallet.send_to(main_wallet, amount)
return transaction_hash

[StasFQ]

so how fix that?

[oguzok2012]

@dgtlmoon how are you?)