Making BIP-39 Seeds Smaller

I've been thinking about a way to shrink BIP-39 seeds into a compact string that fits nicely for QR or RFID storage. Here's the idea: We can turn the wordlist into line numbers. Then, we base58 encode the line number corresponding to each word, making every word become exactly 2 characters long (with zero padding if needed). We can also choose to add a + sign at the end of the output to show whether the seed has an unknown word added or not. For example: Take a seed generated with the BIP-39 tool. When turned into a QR code, it creates a huge code that low-res cameras have a hard time reading. It ends up being a long string with around 102 characters. But with this base58 word-to-line-number compressor, we can shrink it down to a 30-character string, making it way more manageable and readable for QR codes. I even have a sample Python code for this (you'll need tkinter and the wordlist saved as english.txt in the same folder as the .py script you can find the wordlist on BIP-39's GitHub). P.S. Sorry for jumping in with a new account, but I can't find my TOTP code and honestly don't remember setting one for my old account.

A BIP-39 phrase is an encoding of a binary value. If you want to "compress" the phrase, you can use the binary value instead. It can't be compressed any more than that. If you want text, I think base-64 is a better choice, because it is more compact and more widely used.

This is not really compression, BIP39 words are already an encoding of entropy plus checksum. Each word is basically an 11-bit index into a 2048-word list, so replacing the words with line numbers and then base58'ing those numbers is not squeezing magic juice out of it. You are just moving from a human-readable standard format into a custom format that future-you and/or tired-you will have to remember how to decode. For QR use, sure, representing the raw index/entropy data directly can make the QR smaller. That part is reasonable. But I would not casually store this on RFID/NFC unless the threat model is "I hope nobody waves a reader near my backup." Seeds are not coupons. Anything electronic that can be read conveniently can usually be stolen conveniently too. The bigger danger is making a clever backup that only one Python script understands. Wallets understand BIP39 words. Humans can check BIP39 words. Recovery tools understand BIP39 words. Your custom base58 seed-string with a "+ means unknown expanded word" flag is the sort of thing that feels elegant now and becomes archaeological pain in 2031 when you are trying to recover coins from an old drawer and a half-dead laptop. If doing this for fun, fine. If doing this for actual funds, keep the real BIP39 phrase backed up in a boring, standard, preferably non-electronic way. Boring survives. Clever often needs tech support.

That's some valid criticism, let me address it. We are talking about compress to human-usable format, not binary formats. The computed seed for that textual representation is ...or decimal 7377687107399363427919506940445526085943085060767798863389917008452289517492375 9375047662272726680500679881548796212384941982210304075203137347288298516924490 7766500354732069958083853735531831032887267586702456712895290533588458780 This has no use for humans, it's the computer technical part. Even base58 is too much, base46 would be the ideal base size [sqrt(2048)]. Base64 wouldn't give any advantage, the whole idea is to compact a number in maximum of 2 chars. You can even do it by hand with pen and paper, sorry for use GTK in the sample Pyhton, all you need is to split it by pairs, convert it from b58 to decimal and check what line number it corresponds to in BIP-39 wordlist, even Windows Notepad can show line numbers, and if you open in Github instead of downloading the file, Github UI already shows you to the line number. As for the "+" it's to remind you that is not the whole seed, the whole purpose of add a custom word to the derivation path is so that if someone gets your seed backup won't be able to get your funds, but you can write it like:

They are talking about the entropy that the words represent, not the binary seed that's computed from it. In your example, those 15 words represent a considerably short 160-bit value (excluding the checksum). In Hex:in Base64: Since you mentioned BIP39 tool, you can find it by toggling "Show entropy details" after typing/generating the seed phrase.

You don't have to compress that many bytes. Since the dictionary size is 2048 words, each word can be encoded in 12 bits of information. For example - abandon: 0x0000 (using 0-based indexing). For 12 words, that's 144 bits = 18 bytes. So you only need to encode 18 bytes, which is really small, but it assumes that the wordlist is constant and cannot be changed. Thus it will break for e.g. Electrum seeds. If you want to make it human readable, you'd use Base64 which adds about 33% more length.

Base64 is NOT human friendly, that's why Satoshi used base58 for Bitcoin. Also you would gain nothing with base64, other than a potential confusing dictionary. Be it base64 or any other base starting from base46, you will always require two chars. Bitcoin base58 alphabet ordination is not only well known as this tools is meant to be used with it, so if base58 disappears your bitcoins and seed won't have any use whatsoever. The only uptick would be a base2048, in which you would be using only one char, but our alphabet clearly lacks characters enough for it. (And actually that's what a seed is, a number in base2048 format) So, please, let's stop with b64, it's not a "standard" is just the highest base you can go in geometric progression of 8 with the ASCII table printable characters (you could probably go to base128 using the expanded table, but adding odd totally unreadable symbols), but that's it. Good for machines, meaningless to humans, and when comes to this finality we want it to be human usable and as simple as possible - that's why we use words and not simply two bytes stuck together in hex format. By just using the entropy base58 you would get "U1MVu8LMiXXMUKqBUtuCZ3ZrUaV", 1 char shorter than base64. The issue is, using entropy will disable the capacity of doing it with pen and paper. As for "other wordlists", sure, but as long as it doesn't exceed 3364 words, base58 pairs can handle it. You probably will need the software used to generate the seed, if not pure BIP-39, for get its wordlist, just that.

There's absolutely no problem with using other bases, since the idea of such a thing is to make them sharable by QR codes. Users are never actually going to read the string anyway. If they want the human-readble version, well that's why mnemonic words exist.

You wrote: Neither of those are "human-usable" formats. Base 64 is a standard: RFC 4648: The Base16, Base32, and Base64 Data Encodings

Discussion is going sideways for no reason. Let's sum this up: What's the intent? Provide more options to store a Bitcoin (or BIP-39 altcoin) seed. A seed is the weakest link of security, and often induce users on error, such as think that if someone gets the seed but not his gizmo (trezor, ledger, whatever) will not be able to take his coins, others think that reordering the seed or not write one word will make any difference... and ultimately simply write it down in a piece of paper makes that whole security of your "air-gapped, unhackable, 512bits encrypted device" depends on how secure that piece of paper is. QR's and RFID (NFC) Those formats aren't human "immediately" readable (you can learn how to read a QR using your eyes anyway), but nowadays technology made those readers widely available; your mobile from 2015 can read a QR, your mobile from 2020 most likely can read an NFC tag. You can print a QR to paper, but you can also engrave it to metal or wood... there are a lot of options to work with these formats nowadays. However, those supports often just support a bunch of bytes, RFID tags often as low as 64 bytes, QR codes, as you encode more data, become increasingly hard to read or require a much higher printing definition or size. About bases You can work with any base at all, as higher the base the lower the number of symbols to represent a number, however we run out of symbols quickly to further increase it and as we go up we get in troublesome symbols (base 64 includes / - making it unsuitable to write filenames at any *nix based FS, and ambiguous characters, as O and 0), the RFC doesn't mean "it's a standard that everyone should use", the RFC is to set the alphabet (the symbol orders) as we pass 9 (in which case isn't even so as b64(9) = 67). We have the decimal system (base10) as convention for numerals, so to translate any base to another we need to know the proper place of that symbol. Satoshi got up to base 58 as the upper most usable by humans using the western alphabet, base64 is way too much machine only, including an useless to humans pad terminator (=). This said, there's no way to get to base2048, in which case each line number would be a single character, using our set of alphabet and symbols, the ideal base would be base46 as the square root of 2048 is 45.25..., in which case it also be 2 characters per number, but taken base58 is already there it fulfills the purpose. The human decoding Despite the need of a piece of technology to decode the QR or tag contents, this extracts simple readable text, the further work (base58 to decimal to line number on the wordlist) can be done totally offline and with a pen and a piece of paper. Final considerations Even thus the system "complicates" a bit the process, it still offers alternative storage systems, specially non-human eye obvious, such as write down in a paper and put to a vault. Also to note that the seed backup isn't to be interacted with on regular basis, on which this process would be a pain, but just as last resort due to hardware failure (trezor, ledger, etc, bricked).

I highly doubt that! If your camera is so terrible it can't read this, making it smaller isn't going to help much. It sounds like a solution looking for a problem. Custom implementations like this is how someone risks forgetting how to recover his coins. See [overview] Recover Bitcoin from any old storage format for examples of what can go wrong. That's what I was thinking. Seed phrases are designed to prevent mistakes writing them down, there's no reason to use them "on a data level".

This is actually an interesting experiment. My old iPhone 6s camera has a damaged oscillator, or whatever it's called, and this causes the lens orientation to violently shake while the shutter is active. Makes it a pain in the ass to scan QR codes as they keep moving, but it does have better luck scanning smaller codes than large ones. If there was a way I knew about to measure recognition accuracy, it would help evaluate the best method for storing data (and specifically bitcoin secret material) in a QR.

I'd argue  you shouldn't be scanning private data on a mobile device in the first place. Those things are much harder to airgap than a PC, and also much harder to properly wipe. That makes QR-codes one of the worst storage systems for private data.

This has nothing to do or compare to air gapped PC's or even airgapped mobiles or Raspberry Pi's. That's a total different scenario. As for wipe... some folks etch their seeds into metal plates, want something harder to wipe? One improvement I'd been thinking was to use working TOTP keys as backup codes, this provides plausible deniability and camouflage to the seed. But didn't test the concept yet. This also means using base32, so 3 chars per word.

If you can't recover your seed in a safe environment, your backup is flawed. Metal plates don't come with an internet connection But just in case, I can wipe a metal plate in about 30 seconds with my grinder. I could do the same with a phone, but it's much more expensive to do.

Oh, wasn't understanding what you mean with air gapped, thought you were talking about air gapped TX signers. You don't need a mobile, I mention them because they're the most common thing around, yet, without resourcing to mobiles: For read QR codes A webcam + QtQR (cost of a webcam starts from around $5 - thus this is normally a 480p webcam and you will immediately find why the QR size matters). A 2D Barcode scanner + any text editor; cost around $15, those scanners, that you normally see in supermarket cashiers, are just "fast" keyboards, they just type really fast the barcode contents. Your own eyes, if you want to waste time know how to read a QR - that's just an alphabet, a bit complex one, like chinese, but you can learn to read it - , and here size will count even more. For RFID (NFC) An USB reader will be around $10 and a reader/writer around $20. For software there's nfc-tools For destruction QR's are printed in common paper, use a paper shredder. If you happened to engrave them in metal or wood, burn the wood and make sure it completely turned into hash or melt, burn with acid or shred the metal (metal shredder cost: > $1.000). RFID, make sure to break its chip and antenna. *metal option would be the same if you engrave the words themselves, except you'll have a smaller area to burn with acid, if that's your option. QR and RFID also don't come with internet connections, nor the internet is your only enemy when it comes to store a backup.