Evolution of Bitcoin Wallet Formats by Year

I'm trying to compile a list of all the times Bitcoin Core has updated the wallet format. Used some AI for this, so if anyone can verify if my info is spot on, that'd be great: 2009 Non-HD Wallet Version: Bitcoin 0.1.0 Format: The original wallet.dat format. Description: This format saved private keys and transaction details all in one file, without any hierarchical structure. Users had to handle their keys by themselves, and there was no key derivation support. 2012 Non-HD Wallet (Updated) Version: Bitcoin 0.7.0 Format: Still used the non-HD wallet format. Description: While the basic structure stayed the same, there were some updates for better security and performance. However, it still didn’t include the features that came with hierarchical deterministic (HD) wallets. 2014 Introduction of HD Wallets Version: Bitcoin 0.9.0 Format: HD Wallet (BIP32). Description: Bitcoin Core brought in HD wallets, which let users create a tree of keys from one seed. This made key management easier and more private, as you could generate new addresses for each transaction without juggling multiple private keys. 2016 Enhanced HD Wallet Features Version: Bitcoin 0.13.0 Format: HD Wallet with BIP44 support. Description: This version introduced support for BIP44, setting a standard for multi-account hierarchical deterministic wallets. Now users could handle several accounts within one wallet, boosting organization and privacy. 2018 HD Descriptor Wallets Version: Bitcoin 0.16.0 Format

i created my wallet in 2011 and i still use it today. it is still  in Non-HD format. turning an older wallet into a HD wallet is a manual process.

It depends on what you call a format change. There have been several changes to how things are stored on disk with fields being added or removed, but for the most part not noticeable to the user. There have also been new features introduced which are backwards incompatible. This is a non-exhaustive list of serialization changes in the wallet file, including changes to the entire wallet file itself, and changes to specific records. I've marked which ones required explicit upgrades and how to do that upgrade, and which did automatic upgrades for the user. Also tried to mark compatibility. This list is compiled primarily from my memory where I've had to dig around to find out why some vestigial things remain. As such, there may be some missing details if they were removed before my time working on the wallet. Lastly, I've only noted the major version numbers. A couple things were backported to minor versions, these have not been marked. 0.1.0 - Original Berkeley DB 4.7Random uncompressed keys 0.3.17 (ish, a little bit hard to pinpoint the exact version) - Broken serialization of some wallet transaction fields Serialization of CWalletTx changes. Some fields are moved and new fields added 0.3.18 - Reverts previous wallet transaction fields change, with automatic upgrade to undo the change 0.4.0 - Encryption and BDB Update Slightly broken private key encryptionBerkeley DB 4.8 - could cause downgrade incompatibilityEncrypted wallets cannot be loaded in previous versions 0.5.0 - Broken encryption fixed Encrypted wallets last loaded in 0.4.x would be automatically upgraded and fixedEncrypted wallets can be loaded in 0.4.x but will need the automatic upgrade again when loading in newer versions 0.6.0 - Compressed pubkeys Newly generated keys are compressedUpgrade performed with -upgradewallet, old wallets untouched, new wallets have the featureIntroduction of separate wallet version numbers for features 0.12.0 - Removal of wallet transaction merkle branches New CWalletTx records do not have a merkle branchbestblock_nomerkle added to indicate usage of CWalletTx without merkle branchesNo automatic upgrade unless a rescan occursDowngrade forces a rescan for compatibility 0.13.0 - Single derivation chain HD wallet New wallet version number - newly created wallets cannot be loaded on older versionsExisting wallets cannot be upgradedDerivation path is m/0h/0h/*h 0.14.0 - Split derivation chain HD wallet New wallet version number - newly created wallets cannot be loaded on older versionsNo existing wallets could be upgradedDerivation paths are m/0h/0h/*h and m/0h/1h/*h 0.16.0 - Removal of default key defaultkey record is no longer written or used by the software in any capacityProbably has some incompatibility for newly created wallets being loaded in older versions, but that was never really testedExisting wallets are unchanged 0.17.0 - Upgrade path to HD and HD chain split; Optional wallets without private keys New wallet version number - newly created wallets cannot be loaded on older versionsNon-HD and HD wallets can be upgraded to use HD. Upgrades to 0.13.0 HD wallets possible, upgrading to 0.14.0 with chain split not possible. Upgrading to HD chain split requires upgrading to 0.17.0Introduction of wallet flags fields (coincides with version number bump that was also needed for the HD upgrade)New wallet type with a flag indicating the wallet does not have private keysWallets without private keys are incompatible with older software 0.18.0 - Key metadata with binary origin info; Optional blank wallets Serialization of key metadata changed to support binary origin info, upgrade to include this info done automatically, old info kept and written for downgrade compatibilityNew wallet flag to indicate the upgrade was performed to avoid doing it every timeNew wallet flag introduced for blank wallets. blank wallets cannot be opened in any prior version, although once things are added, they can be opened in older versions (that support wallet feature flags) 0.19.0 - Optional wallets that avoid reusing addresses A new wallet flag to indicate that the wallet should avoid reusing inputs associated with addresses that have already been spent fromExisting wallets can add this feature with setwalletflag, and wallets with the feature can turn it off with setwalletflagWallets with the flag set can be downgraded but the flag will not be enforced by older versions 0.21.0 - Optional descriptor wallets using SQLite New wallet flag indicating that the wallet supports using descriptorsDescriptor wallets are optional and no the default. Newly created descriptor wallets use SQLite for the database rather than Berkeley DBDescriptor wallets cannot be loaded in older versionsAutomatically generated descriptors use BIP 44 derivation paths 22.0 - Descriptor wallet cache change; Optional descriptor wallets with external signer (hardware signer) Cache records for descriptors is changed, and new records addedExisting descriptor wallets automatically upgraded in the backgroundNew wallet flag indicating that the upgrade was performed to avoid redoing it in subsequent loadsUpgraded wallets can be loaded in older versionsNew wallet flag to indicate that a wallet should query an external program for descriptors and signing of transactionsWallets with the external signer flag cannot be opened in prior versionsUpgrading to external signer is not possible 24.0 - Legacy wallet (non-descriptors) to descriptor wallet migration tool which requires explicit user action to migrate 29.0 - Legacy wallets can no longer be created 30.0 - Legacy wallets can no longer be loaded at all Here's also a list of a couple other things that I remember occurring but don't remember when they happened: The encrypted private key records were updated to include a checksumThe accounts system was replaced with labels, but IIRC no fields were changed. Several just became unused and no longer read when loading, and others were repurposed for the labelsThere used to be some records for the market thing that Satoshi originally wanted to have, but they were never used/were removed pretty quicklyAnother field in CWalletTx was deprecated at some point because it was no longer being used

So how do you manage your wallet, do you have a 2 laptop setup with a watch-only wallet? Also, what will you do once Core 30 deprecates your wallet? Im using Knots 28 so I don't know what they will do in this regard.

on core 27.1 here i just have a bog standard wallet created in mid 2011 and i just did in place upgrades from there. when the original HD format was introduced i elected to keep my old wallet instead of creating a new one as i had mining payout addys in it and exchange whitelisted addys etc in it and, as im lazy af,  i just kept the old format and made sure i always backed up wallet.dat to make sure any new keys generated would my in my backup set. now with core supporting multiple wallets i can make a new HD format and use it in parallel BUT im pissed that legacy wallets wont be supported from 30 on?? wtf satoshi said never lose control of your private keys... so i guess  ill run a final pre 30 version and run it in parallel on a dedicated machine.

They aren't supported, but the migration tool is built into the software and will seamlessly (in the vast majority of cases) migrate your legacy wallet into a descriptor wallet.

makes old man shakes fist at sky motions *hrmpth* perhaps ill try it out now. more seriously one aspect of bitcoin that has cemented it in my mind as the most bullet proof way to store value there is is the fact stuff i created 2011 is drop in compatible now. im very cautious about change.. note im a couple major versions behind.. heh i still use legacy addys

Do you mean the existing migration tool or the one updated in version 30? Im also interested in knowing what version my wallet is. I reckon it must have been around 2013, but im not sure what actual current wallet.dat stuff is on the file right now. Is it possible to find out?

for HD vs Non-HD import to core. when core is running look at the bottom right of the UI, there is an HD icon, mine is greyed and lined out, so Non-HD. if its lit (or not lined out) its HD.. afaik anyway as for specific versioning dunno.. edit windows versions for the most part

Yeah, I remember the icon being greyed or lined out, so it's an old wallet for sure, however, im not sure if further Bitcoin Core upgrades may have modified the wallet in some way. I do not remember ever performing any manual upgrades to the wallet file, so im not sure if it's the same as day 1 from it's creation, or some modifications were made on the file.. achow mentioned some upgrades on that list, however, im not sure if that list means every time a new wallet format was introduced, but I don't know if that includes automatic upgrades to existing wallets or not.

It's the same migration tool, nothing has significantly changed in its operation. You can get the wallet version number from getwalletinfo in the walletversion field. However, that won't include any information about automatic upgrades which are not versioned. Once v30.0 is out, there's a new flags field in getwalletinfo which contains a list of all of the wallet flags and that will tell you about which features your wallet specifically supports, including any of the automatic upgrades which set a flag. However, not all automatic upgrades set a flag. The only way to detect those is to dump the raw data from the wallet itself and inspect the records.