Issue with importdescriptors command

13 replies 168 views
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#1Jun 19, 2023, 03:47 PM
I'm testing with a testnet3 wallet and ran into a problem when I tried to use the "importdescriptors" command after pasting content from "listdescriptors false". I keep getting a bunch of errors. Anyone know how to get around this? I have another watch-only wallet created from the same testnet wallet earlier, and that one worked fine. Now, I tried importing from the same wallet to set up a new watch-only wallet, but the error popped up again. Could this be related to using pruned mode? Back then, I had the full blocks, but now I don't seem to? It’s weird, some funds show up, but most don’t. Maybe using some rescan commands can help? I want to stick with pruned mode on testnet since I don’t want it taking up extra disk space, but I’m worried I might end up needing the full blockchain anyway. Seems like pruned mode just causes issues (the migration wallet bug was a good reminder not to use it). If I have to do a full resync of the blockchain every time I use importdescriptors, I might as well just go with non-pruned mode... so what's the best move right now? Also, if I need to download the entire testnet3 blockchain, maybe I should switch to testnet4? I’ve heard testnet3 might be phased out soon, but I’ve also seen some mentions of exploits on testnet4. I really don't want to allocate a ton of space to testnet if it’s just gonna get spammed or something.
6 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#2Jun 20, 2023, 04:47 AM
Yes, it does. There's a "start_height" arg in rescanblockchain command that you can use to scan your oldest block, But that wont solve the issue of missing transactions in the already pruned blocks. The alternative that I know is importprunedfunds but it requires a "txoutproof" from another testnet3 node. If the datadir's size is the issue, then certainly yes. For example, my testnet4 node's entire datadir including wallets is currently only 13GB in size. Or you can use Regtest, you'll only need about less than a GB for its datadir. It's perfect for testing PSBT workflow.
2 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#3Jun 21, 2023, 03:41 AM
I don't know how regtest works, never heard of it. I prefer to test things live, it gives me more peace of mind that something works imo. So I will try testnet4, if it's only 13GB It's no big deal. Where can I see how big the testnet3 and testnet4 blockchains are? I cannot even see the maintest size on mempool.space site. Btw, do you know a legit site to get testnet4 coins?
0 Reply Quote Share
im_lynxHero Member
Posts: 515 · Reputation: 2161
#4Jun 22, 2023, 07:20 AM
Regtest is as live as you want it, you control block creation yourself. Good luck with the blocks forking nonsense and some stupid miners on Testnet4 that don't care to include transactions from mempool(s). Oops, it actually got a bit better from the last time I checked on https://fork.observer, nice! You could learn how to use Regtest, it's not too hard. I'm sure there're good explanatory posts in this forum, just a little search away. A working faucet for Testnet3 and Testnet4 e.g. is: https://altquick.com/faucet/, but I can send you Testnet4 coins, too. Just state how much you need (don't be too greedy) and your Testnet4 public address.
5 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#5Jun 22, 2023, 11:03 AM
For that workflow, Regtest has it all covered. It follows most consensus rules related to your intended test from PSBT generation to broadcast. The only difference is its network is limited to what you can connect to it but that's not the main point of your test, unless it's about transaction propagation speed which testnet/testnet4 also can't accurately reproduce the same result as mainnet since those consist of different set of nodes. Here's my testnet4 directory's blocks folder if you want an example: Approximately 12.8GB in size.
4 Reply Quote Share
paul.ninjaFull Member
Posts: 152 · Reputation: 539
#6Jun 22, 2023, 03:40 PM
What is biting you here is not really importdescriptors itself, it is the rescan window colliding with pruned history. Core rescans from the descriptor timestamp minus a little safety margin, and if those blocks are already gone, there is nothing for it to read. A lot of people hear "try reindex" and think that magically summons old blocks back from the grave. It does not. Reindex can only work with data you still have on disk. If pruning already ate the relevant range, the wallet is trying to remember a movie after somebody burned the first half of the reel. One useful workaround, if your goal is to see what is still unspent rather than rebuilding your full wallet history, is to import the descriptors with timestamp set to now so it skips the historical rescan, then run scantxoutset against those same descriptors. That scans the current UTXO set, so it can still find live coins without needing old block files. It will not reconstruct old transactions or labels and all the cozy wallet history stuff, but it is a decent way to sanity check what is actually still there on-chain before you decide whether a full unpruned rescan is worth the hassle. One other thing I would double-check, because it can produce the exact "some funds show up, most don't" headache, is whether you imported the full descriptor set including the internal/change descriptors and the correct range metadata. Pruning is the obvious suspect here, yes, but a too-small range or missing change descriptor can quietly make a wallet look half-empty even on a healthy node. Also, if you are choosing a public test network for wallet workflow testing, I'd seriously consider Signet. Regtest is great for deterministic local work, Testnet is chaos with a pulse, and Signet sits in the nice middle where the network actually behaves like it had breakfast.
0 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#7Jun 22, 2023, 07:27 PM
I have refused to use any pruned stuff, it is really a waste of time. Testnet4 is small enough that you can work with it reasonably fast. Not sure what signet is, will check it out. The big problem with all of this anyway is, watch-only wallets are unable to order transactions in proper order of occurrence, because it dumps it all as if they were all received at the same time at least if they belong to the same "timestamp" value.
6 Reply Quote Share
luckyapeFull Member
Posts: 77 · Reputation: 599
#8Jun 22, 2023, 10:38 PM
Yeah, the important bit here is that watch-only ordering is not magic, it's just Core reconstructing wallet state from what it can actually see. If the relevant historical blocks are missing because the node was pruned, then it can't properly discover the old transactions no matter how correct the descriptor is. The descriptor tells Core what to look for, it does not summon old block data from the shadow realm. For this kind of testing, I'd stop mixing pruned mode with descriptor import tests. It creates exactly this kind of half-working swamp where one wallet seems fine and the other behaves weirdly because their birth times, ranges, change descriptors, and rescan windows are not identical. Testnet4 is small enough that I'd just run it unpruned and remove that variable entirely. Life is too short to debug missing archaeology. Also, if you're dumping descriptors from an existing wallet and importing them elsewhere, make sure you import the full set, including internal/change descriptors, with proper ranges and timestamps. A lot of people import only the receive side and then wonder why the wallet acts like it was hit in the head with a shovel when change outputs enter the story. Use listdescriptors true, inspect what you're actually moving over, then import the whole family, not just the pretty cousin. If you only care about future receives, timestamp now is fine. If you care about old funds and ordering/history, use an unpruned node and rescan from a sane birth time. Otherwise you're basically asking a wallet to remember a movie after you deleted half the film reel.
0 Reply Quote Share
yield_forkFull Member
Posts: 162 · Reputation: 728
#9Jun 22, 2023, 10:55 PM
Try importing the descriptor again, but in the timestamp, always include "now" so the software ignores the rescan. From there, try importing the funds with the importprunedfunds command (it should work even if your node isn't pruned). With a quick search for more details on how to use importprunedfunds, I found this post below: If your target transaction is within the reach of your pruned node, you can obtain the raw transaction directly from your node using: getrawtransaction "your_txid"
4 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#10Jun 23, 2023, 02:23 AM
Take note that this command will only work if that particular transaction is still in your mempool, For transactions that are already in the blockchain, it requires txindex or the block hash where that transaction is included. For nodes with pruned blockchain that can't have --txindex, the command should be like this: As you mentioned, the block should till be available for it to work.
0 Reply Quote Share
yield_forkFull Member
Posts: 162 · Reputation: 728
#11Jun 23, 2023, 02:34 AM
yeah, I had completely forgotten about the need for -txindex enabled.  It sounds so natural to me that I've been using this command for so many years with txindex enabled. I didn't know that we can also query transactions with getrawtransaction on a pruned blockchain without -txindex, as long as it meets the aforementioned principles. This makes me see some of the advantages of a pruned node (which I didn't see before). I intend to run my full node until I have the budget for SSD storage, I'll be sad if one day it's no longer financially viable to run a non-pruned full node.
3 Reply Quote Share
im_lynxHero Member
Posts: 515 · Reputation: 2161
#12Jun 23, 2023, 08:06 AM
You can easily put the block files on a HDD (with blocksdir=<path-to-folder-on-mounted-HDD> in your bitcoin.conf file) without much speed penalty and keep indexes and chainstate and everything else on a fast SSD. Large enough HDDs should remain quite more effordable than similar SSDs. In my opinion there's not much need to put the block files on a SSD unless you want to operate a heavily used Electrum server and a full node as blockchain data provider for many Electrum clients.
0 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#13Jun 23, 2023, 01:28 PM
Yeah, providing the block hash eliminates that requirement because it doesn't have to scan the blockchain for the particular block. By the way, since the goal is to use importprunedfunds on a node with pruned blockchain that failed to fully rescan due to some pruned transactions. Using getrawtransaction on it doesn't make sense since the target transaction is already pruned. If that's still in his blockchain, then he can just rescan starting from that block or earlier unpruned blocks.
3 Reply Quote Share
yield_forkFull Member
Posts: 162 · Reputation: 728
#14Jun 23, 2023, 02:01 PM
Thanks for the tip, I've done that before when I bought my first 2TB SATA SSD (later installed it in my laptop). But while it was outside the machine, I was using it as an external SSD and using -datadir, it worked very well at the time, it served me well for a while. Regarding using an HDD instead of an SSD to store the blocks, it might be a good idea, but I'm afraid I might accidentally disconnect the USB cable from the external HDD/SSD and cause data corruption, but anyway, I'll make the decision when the time comes. Yes that's true, but we can't fully rely on rescanblockchain command, i.e, if you have many transactions, there's a possibility it won't see all the available UTXOs. This happened to me recently in v30.2, I had to rescan again for bitcoin-qt to see the missing UTXOs. That's why importprunedfunds might be more straightforward, although it's more complex for beginners to use. An extra tip for using rescanblockchain: always set a slightly larger range for the start_height and stop_height blocks of your target UTXOs. This might increase the chances that the rescanblockchain command won't miss any available UTXOs (unless you want to use a full rescan, which can take a long time).
4 Reply Quote Share

Related topics