Need help recovering legacy wallet.dat (bounty offer)

So, I created a wallet.dat using Bitcoin Core back in 2014 or so (pre-HD). I bought Bitcoin from exchanges a few times and kept it in that wallet. It's encrypted, and I've got the wallet.dat file along with its passphrase. Here's the issue: I didn’t realize that you need to back up wallet.dat whenever you spend Bitcoin from it. I thought it was only necessary to back it up after receiving additional coins. I had my Bitcoin Core and wallet on a dedicated Linux laptop, but after updating to a new Linux distribution, I lost the working copy of wallet.dat. I only have the original version left. Once I figured out my mistake (like copying the original wallet.dat to a new computer and finding it almost empty), I created a disk image of my laptop's drive using a live Linux USB and the dd command. I then tried various file recovery tools like PhotoRec to retrieve files from that image. Now, I’ve got a couple of recovered files that seem to be the lost, updated version of wallet.dat. These restored wallet files are around 1 MB, while my original wallet.dat is about 115 KB. I checked them with a binary comparison tool, and the recovered 1 MB files share the same content for the first 115 KB, then have new content that doesn't look totally random it resembles the original wallet file's contents.

I don't think that's entirely accurate. It was for the earliest version of Bitcoin-qt, but they quickly added a 100 key "pool": the wallet.dat would create 100 unused private keys, so your backup would still be valid for a while. But if you set or changed a password, those 100 keys would be replaced by new ones and a new backup was indeed required. Each time you sent a transaction, the change was sent to a new address in your wallet. So it's kinda like receiving coins. Instead of using other recovery software first, have you tried to use pywallet on the partition (image) directly? See [GUIDE] Recover your deleted keys.

Indeed, this is as much as I learned after I started to figure out what could be the reason why my balance was showing almost zero. And I have not made many transactions with the wallet to begin with. But nevertheless, Bitcoin-cli shows almost zero balance. I tried to do "bitcoin-cli -rpcwallet=legacy rescanblockchain 0" as well, to see if that would somehow fix the situation, but it didn't help. Running bitcoind on the original wallet.dat says: EDIT: I have not changed the password of the wallet since originally creating it. I have not tried that. Does the pywallet's --recov_device parameter accept a path to the disk image, or do I do something with the disk image first, such as mount it?

It probably has an option to use the image instead of partition, or maybe it just works (Everything is a file): Alternatively, you could just use the partition on which you saved the image, but that may be much bigger than needed.

At least there was no instant error message when trying to do that, so I now have pywallet with the recover option running, with --recov_device pointing to the image file from the dedicated Linux laptop that hopefully contains some private keys. The image file is about 1 TB in size, so this will take a moment to run. I will report back when it has completed. Thank you very much for your help so far.

A slight correction, changing the password of an already-encrypted wallet wont set a new keypool. Or was it different in the first implementation of wallet encryption in version 0.4.0? Please CMIIAW. You can also try to paste a copy of those corrupted recovered wallet.dat to a small partition that you'll set as --recov-device and see if pywalet --recover can find private keys in those.

I'm not entirely sure, all this happened before my time in Bitcoin

I ran it with the --recov_device pointing to the path that contains the dd created disk image file. It ran and created a "pywallet_partial_recovery_xxxx.json" file. It didn't create a new wallet.dat file, as it said it would. Any tips what should I do now?

I can't really tell you, I haven't had to deal with this in a long time. But I'd start by reading what's in that file (and don't post it of course).

I also ran the pywallet --recover directly with drive instead of the disk image input, and after it ran, it outputs: This output seems to indicate that it does find something, but for some reason crashes at the very end. The json files that it outputs are 30+ KB in size, they contain things like mkey: [xxxx,zzzz], and ckey: [qqqq,yyyy] etc. Where the data itself seems to consist of 12 digit numbers. Something like this: ckey: [194857392812, 3948595949192, 384759284843] and so on. To me, the mkey sounds like it is a reference to the wallet's master key and ckey sounds like the encrypted private keys.

Which version of Pywallet did you use (source link)? Because the guide [GUIDE] Recover your deleted keys tells you that Pywallet from thread Pywallet 2.2: manage your wallet [Update required] needs Python 2.7 and you used Python 3.13. In jackjack's Github https://github.com/jackjack-jj/pywallet I don't quite see yet support for Python 3.x. This post from nc50lc mentions above latest version of Pywallet to work properly with Python v2.7.17.

I'm using pywallet from jackjack's Github. You are correct, I was using Python 3.x to run it. I have now installed Python 2.7 via pyenv, and re-ran it, and it produces the exact same output, I'm afraid. It seems to crash while trying to save the possible wallets.

Since you've said "exact same output" in your previous reply above, Does it include this quoted error line: ? Because even though v2.7 is installed via pyenv, it may still be using your python v3.13 environment to run Pywallet. If so, try this pyenv tutorial to set it to v2.7.x: realpython.com/intro-to-pyenv/#specifying-your-python-version

Yes it does, I found that Python 3 reference to be a bit odd, too. But when I ran "python --version" before running pywallet, that does say "Python 2.7.18" I shall give this a read. Thank you!

I started writing a modern pywallet replacement that may work: https://github.com/achow101/wallet-manipulator. You should be able to install it with pip install . and then the wallet-manipulator command should be available. You can then export your private keys with You can use an additional --importable option after export to get json formatted output that can be dropped directly into a Bitcoin Core importdescriptors command. However, since you said that these files are recovered rather than the original files, this may not get all of your private keys nor will it necessarily be able to even read the files. It uses my own implementation of a BDB file parser and it generally requires that the file is properly formatted. In possible corruption scenarios, it may not behave as expected. You may want to try first to see if it is able to parse the file and get any records out of it.

Wonderful! Eager to see the final and practical result of this implementation. I also had difficulties running Pywallet due to this confusion with Python versions and other errors that I don't remember (it's reported somewhere on the forum). I have a question: will this wallet-manipulator implementation workfor recovering descriptor wallets as well or only legacy wallet.dat files? Do descriptor wallet.dat files store any WIF private keys? In other words, can I restore WIF private keys from a descriptor wallet without needing third-party tools such as web pages (iancoleman)?

Yes, it can export from descriptor wallets. Not in a way that you would expect or would get useful results from. The default mode of operation for descriptor wallets is to have a descriptor which contains a xprv from which the keys for the addresses in the wallet are derived. However, the derived keys are not stored in the wallet like they are for descriptor wallets. Instead, they are derived from the xprv when needed. The xprv itself is not actually stored as an xprv. The actual private key component of the xprv is stored separately in the same format used for storing private keys that legacy wallets used. The xprv is essentially reconstructed at derivation time by taking the xpub in the descriptor and swapping out the pubkey for the privkey. So if you were to parse the records of descriptor wallet naively, you could create a WIF key that corresponds to the private key component of the xprv. But this WIF key would be absolutely useless to you as it is not directly used as the private key for any of your wallet's addresses, and it is not enough information to derive any of the actual private keys (you need the chaincode from the xpub to do derivation). If your wallet has a descriptors for things that have the key directly in them rather than deriving keys, then yes, you would be able to get the WIF and have them be usable. But this really only happens if you import those things yourself, or if you migrated a pre-HD legacy wallet. wallet-manipulator won't be exporting the keys for descriptors as WIF. It will instead reconstruct the descriptors with the full xprv so that users don't shoot themselves in the foot.

I tried to run Pywallet via pyenv but it always crashes with the previously mentioned error message. Since I wasn't sure whether this is some kind of Python version related issue, I also installed an older version of Ubuntu with Python 2.7 and tried it within that system as well, but no, Pywallet just always crashes. Since this approach seems the most promising (Pywallet does say that it can detect wallet and private keys from the recovery data that I show to it), I decided to try to pursue this path a bit further. I'm probably going to dox myself but the thing is that I run a software company and I got a Python developer. So I decided to use some of my own money and I assigned him to update Pywallet. He already completed the Python 3 support, but even with that implemented, the crash still happens during the recovery attempt. I'll have him try to improve the recovery feature of Pywallet in a way that in case of too corrupt data, it will recover anything it can without crashing. The worst case scenario is that I cannot recover my coins, but at least Pywallet gets updated, so at least there's that. I will release it all in Github, of course. I hope I will have an update about this project soon.