Old wallet.dat from 2013 won't load in version 29

I've seen people saying that moving old wallets to new versions is a piece of cake, but honestly, it's proving to be a lot trickier than they make it sound. I tested it out using Bitcoin 0.8.6 on a Tails live session. I got the file from bitcoin.org, and the SHA256SUMS.asc file is signed by Gavin Andresen, so I thought it was legit. I created a wallet.dat file, generated some addresses, saved it, and then transferred it to my fully synced Knots 29 node. When I tried to load it using just the GUI, it got stuck on loading for ages. Then I gave the bitcoin-qt console a shot with the loadwallet wallet.dat command (putting it in the "wallets" folder, which I assumed is where the new versions expect wallets to be) and it threw an error: This action generated these 3 files: db.log shows this: I seriously doubt the file got corrupted since I made it just yesterday. I think this might just be a compatibility issue with older versions. So it doesn't seem as simple as just dropping your old wallet file into the latest version and hoping for the best. Now I'm kinda lost on what steps I need to take to make it work. If you're curious and want to try this out yourself, you can grab the binaries here and create a wallet.dat file, then attempt to load it into a newer version: https://bitcoin.org/bin/insecure/bitcoin-core-0.8.6/ If you can't run it due to missing qt4 libraries, you’ll need to download those: https://archive.debian.org/debian/pool/main/q/qt4-x11/ Specifically, look for these: https://archive.debian.org/debian/pool/main/q/qt4-x11/libqt4-core_4.8.2+dfsg-11_amd64.deb https://archive.debian.org/debian/pool/main/q/qt4-x11/libqt4-network_4.8.2

Please clarify what you actually did, as there is no "save" action. This is extremely unlikely to be a backwards compatibility issue. Looking at the log snippet you provided, it is highly likely that the error is a result of only copying the wallet.dat file. Contrary to popular belief, the wallet.dat file is not the only file you have to copy when copying a wallet from one node to another. You also have to copy that database folder. This is especially true for older versions of Bitcoin Core as shutdown did not always compact everything into the wallet.dat file. Furthermore, copying the wallet.dat without unloading the wallet (or stopping Bitcoin Core) will result in data loss, and therefore detected corruption, because the log files in the database directory contains data before it gets committed into the wallet.dat file. This behavior is also one of the reasons that we wanted to move off of BDB. SQLite can be configured to specifically not do that and to always write into the wallet.dat file.

By saving I meant, I exited 0.8.6 bitcoin-qt, copied the wallet.dat and then pasted it into the hard drive to open it with the knots 29 bitcoin-qt. I've never heard you needed the chainstate files. I may or not have a copy of the chainstate files of the old Bitcoin Core install where the actual wallet with private keys came from. It may still be there in some old Windows install of Bitcoin Core. Im assuming there is a way to salvage the private keys from the wallet.dat without chainstate files otherwise I would be screwed. Im hoping this is a particular thing that happened by doing this on a Tails live session, perhaps something went wrong there. I will try with the real wallet files on the airgap computer just in case, if I get the same error then im going to need some good ideas in case I cannot find the chainstate files from whatever install the wallet was last used in. Would actually be insane you need these files to backup your keys but I will try to find them anyway.

I did not say chainstate files. The files in database/ are not for the chainstate, they are the BDB log files and used for Write-ahead logging. There is no such guarantee. The log files will contain data that has not been written to the database file yet. Depending on when the wallet.dat was copied, the data may not yet exist in the database file and therefore losing the log files is catastrophic data loss. Would be insane to continue to use a database system that requires that.

This may explain why I've seen countless topics over the years from people who can't open their old wallet.dat because it's corrupted.

Yeah I meant the database folder. Well the thing is, I managed to go into the old drive that had the full node, ages ago I was running a Windows node, and I kept updating that one. The last version I have saved is binaries for 0.16 so I assume that is what the bitcoin folder was running. There's a bunch of wallets from different years backed up. The last wallet file is from 2019. What I noticed is that there doesn't seem to be a database folder on the main folder, there is a database folder in the testnet folder but not on the main datadir folder. I don't remember deleting any folders so that's weird. I'll have to test those files, hopefully the most recent backups work.

It depends on the version that was last in use. Newer versions would ensure that all the data from the log files ended up in the wallet.dat file, then delete the database folder, on shutdown. This would also happen periodically during operation. However, older versions and unclean shutdowns would still have the database folder.

Im looking at the debug.log and the last times I opened Bitcoin Core the version it says it was v0.19.0.1 Lines that have to do with the wallet: init message: Verifying wallet(s)... Using BerkeleyDB version Berkeley DB 4.8.30: (April 9 2010) Using wallet C:\Users\takuma\AppData\Roaming\Bitcoin BerkeleyEnvironment::Open: LogDir=C:\Users\takuma\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\db.log some other lines here about blocks ad chainstate like "Opened LevelDB successfully" etc then No coin database inconsistencies in the last 6 blocks init message: Loading walet.... BerkeleyEnvironment::Open: LogDir=C:\Users\takuma\AppData\Roaming\Bitcoin\database ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\db.log [default wallet] Wallet File Version = 60000 [default wallet] Keys: 0 plaintext, 1000+ encrypted, 1000+ w/ metadata, 1000+ total (im typing this by hand looking at the screen so im not putting exact numbers) Unknown wallet records: 1 [default wallet] Wallet completed loagin init message : Rescanning... Here it starts scanning and I have some AddToWallet new So I was opening the node and the date says 2020 so I was still using the wallet there and reeving transactions... but there is no database folder, not even a file inside the database folder. The testnet3 folder has a database folder with log. It also has the "ErrorFile=C:\Users\takuma\AppData\Roaming\Bitcoin\testnet3\db.log thing" But for testnet3 the last version on the debug used was 0.15.1 So I don't get it, as I never did any manual updates of the wallet format. So either the wallet.dat will work without the database thing because Bitcoin Core v0.19.0.1 is modern enough to not need it, or somehow the database folder disappeared. The last lines do not seem like there was an error during closing: addcon: thread exit Shutdown: In progress torcontrol thread exit net thread exit msghand thread exit opencon thread exit scheluder thread interrupt dumped mempool [default wallet] releasing wallet Shutdown: done

As I said, on a clean shutdown, the database folder is deleted by newer versions. 0.19 is new enough.

Sorry for jumping in, but as a much newer bitcoiner (I started 3-4 years ago), what is the database directory? According to @achow101 it's deleted since version 0.19 and onwards, but what was its purpose? What did it include? And more importantly, for newer versions, isn't it true that we only need to backup the wallet.dat files?

This one: He wrote it gets deleted on (clean) shutdown. I have that directory.

I'm running Core 29.1 but I don't have it. I guess it's an old directory that's not used anymore? The reason I'm asking is that I wanna make sure that keeping simply the wallet.dat for backup is enough.

Yes, I have observed this behavior, and now it makes sense. It explains why there is no database folder. The wallets work when I loaded them in Knots 29, with labels and all. The HD logo has a crossing line on it indicating it's not an HD. So the experiment may not have worked because Tails did not properly shut down the wallet and I also did not copy the database folder. But the actual wallet did work thankfully, now I need to plan on a watch-only setup. I will also test the migration tool with some backup. An observation: When I loaded the wallet.dat file on Knots 29, the sha256 checksum changed. What changes are made to the original file? When you load a wallet, you should see a database folder, but when you close it, it disappears. If you don't see it, then im assuming this behaviour happens with dbd (berkeley) wallets and not sqlite wallets (more modern ones) which never need that database file. And perhaps really old ones (2013) but that never went beyond the old versions (because I did updates on Bitcoin Core, I assume some updates were applied along the way on the wallet file as I updated) always need that database folder and it's never deleted? Not sure about that. On the experiment I did with the 0.8.x version, there was a database folder, even after closing it. I did not see any errors, but I think I also didn't the see the "Bitcoin is clossing please wait..." message, but I don't remember if that message was there back then. But doing this on Tails may not be reliable anyway, I just didn't want to install these binaries so used Tails for it.

It is still used if you use a legacy wallet. The directory is always located in the same directory as the wallet.dat file which may be different on your computer. This whole discussion is about how it isn't. Frankly, backing up a wallet by going to the filesystem is not recommended (or really supported) method of backup because of these kinds of issues. The only recommended way to backup a wallet is to use the backupwallet RPC or File > Backup Wallet in the GUI.

Yeah, I figured, thanks for clarifying. That's exactly how I'm doing it (using the GUI version). The output of this command (i.e what's actually saved) is a single wallet.dat file. When I read your post above, talking about backing up the database directory, I felt like asking, just for clarity.