So, LoyceV and I are planning to make all public keys for every address available to the public. I've worked out the process and the coding part, and LoyceV will host it on his server.
We'd love to hear if you guys are interested in access to the public keys for all these addresses.
We're pulling the data straight from the "blocks" and "chainstate" folders of Bitcoin Core.
Making public keys available for every address
18 replies 466 views
sam_walletFull Member
Posts: 104 · Reputation: 365
#2Oct 23, 2024, 08:13 AM
What purpose would a log of all available public keys serve? I'm trying to think of possible reasons anyone would want that but I can't come up with any. Can you share what prompted the project?
alexwalletSenior Member
Posts: 347 · Reputation: 1933
#3Oct 23, 2024, 02:00 PM
^^
What immediately came to mind was blockchain analysis/identity clustering, like those performed by chain analysis services. Or perhaps it could be a reference for certain statistical data (who knows).
This is the same as puzzle ( https://privatekeys.pw/puzzles/bitcoin-puzzle-tx ) - maybe someone would be happy to download fresh list of public keys for addresses.
Also there can be other usage which I don't know yet. However ppl are asking for public keys recently on the Internet.
We are in contact with LoyceV currently, I am preparing the file for him.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#5Oct 24, 2024, 07:30 PM
Maybe someone is preparing for the first available quantum computer to crack them all
Then why are you posting on the Dev board?
Yes. What's with the shitposting in this thread?
OP asked me to host the 40 GB file.
Oh noes, someone shares data from the blockchain! Those blockchain analysis companies would never have thought of that on their own!
/sarcasm
As for questioning why: it's data, and I like data:
Whoever wants it, can have it. If anything, you can use it to prove to yourself how secure Bitcoin is. Did you know that every time your wallet creates a new Bitcoin address, it's basically brute-forcing all existing addresses, even if it's just one address at a time? Everyone who ever installed a wallet has attempted to brute-force a funded Bitcoin address and failed.
Then you'd better think about maintaining a database of addresses with non-zero balances and keeping it up to date.
Because the quantum threat is about cracking transaction signatures using Shor's algorithm. It is also related to addresses whose public keys have already been revealed through spending transactions in the blockchain.
The quantum revolution is out, Op and LoyceV are going to lead that race here in the forum. We could all be mega rich here if things go well.
I have seen the data of all addresses having bitcoin and its importance, but I can't figure out the importance of this, except for sophisticated analysis.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#8Oct 28, 2024, 06:42 AM
OP uploaded the data, it's available on http://tmp.loyce.club/pbies.
SwiftMinerSenior Member
Posts: 259 · Reputation: 1036
#9Oct 28, 2024, 08:00 AM
Just based on curiosity, op asked for all public keys with addresses that begin with 1 , how exactly are you able to pull up this data? Is it from the node you run or something totally different? I'm asking because what actually came to mind is public keys of addresses that have actually made a transaction.
Well I may be wrong though but what I'm thinking is you can only pull up Data from addresses that have made at least a single transaction because of the data they share with node... Where could I be wrong though?
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#10Oct 29, 2024, 08:28 AM
OP only asked me to host it, and included the "how" part in the link.
I am doing that like there will be files info.txt and readme.txt, I am pasting them here:
We need "blocks" and "chainstate" folders from Bitcoin Core, fresh, so sync it before doing next things.
Core need txindex=1 in bitcoin.conf file to download chainstate. You may need to start Core with -reindex or something similar.
then (in WSL2 on Windows 10, Ubuntu 24.04 or 26.04):
rsync -avtu --delete /mnt/d/Bitcoin/chainstate/ /home/user/.bitcoin/chainstate/
Change first folder to source, second to target from which you will take data later.
Pubkeys unpacked = 58.24 GiB, packed pigz = 38.12 GiB.
It may spoil the blockchain, Core will need to reindex. Use on copy.
Last: build the below C++ program and run it:
1.
g++ -O2 -std=c++17 btc_addr_pubkey.cpp -lssl -lcrypto -lleveldb -o btc_addr_pubkey
2.
./btc_addr_pubkey --both ~/.bitcoin tmp.tsv
3.
LC_ALL=C sort -S80% -u --parallel=30 < tmp.tsv > pubkeys.txt
4.
pigz -9 pubkeys.txt
###
first file is 139 GiB
sorted file is 58.3 GiB
dump takes 40 minutes
sort takes 31 minutes
pigz takes 9m44s
38.13 GiB has output file compressed
Core's blockchain is spoiled, make backup!
###
This dump's timestamp is for 2026-05-08 09:15:41 CEST.
###
I can share btc_addr_pubkey.cpp file later.
It was Claude AI's work. C++ code should be on LoyceV's site later.
Have you include PubKey from P2PK addresses too ?
Please, do not forget about P2WPKH too!
SwiftMinerSenior Member
Posts: 259 · Reputation: 1036
#14Oct 31, 2024, 05:16 PM
So technically speaking I'm actually on track. Literally the public keys you are gonna be pulling out are only those of wallets that have at least made one transaction so it's like we are kinda treating Wallets with zero transactions as those that haven't been hashed yet or created per say.
This is kinda public information though as it's basically something every full node can pull out. Before now I haven't really thought about this because I hadn't really seen real use cases though. Nevertheless I do understand the whole stuff regardless now.
P2PK is a legacy address with pubkey exposed in the script, P2WPKH are not included like Pbies have already write about it
Example: https://www.blockchain.com/explorer/addresses/btc/1JWXHwtBuVGDDjrVDQNFaBHhw7AhuuPeV9
PubKey: 026843174b9bc9eb6d69016b1b9ec74c38af7f3bf1a153ab7f5d3ea8cd9835ce54
oracle_rocketMember
Posts: 9 · Reputation: 141
#16Nov 1, 2024, 02:35 AM
Nice motive. But the tin be say getting public keys generally for one addresses he go help out for research and also privacy analysis and it also shows the risk of address reuse.
My question be say: una go include only used addresses wea get balance or one addresses that exist on blockchain history ? This because the second will be a huge dataset.
The data is taken twice, once from "chainstate", second from "blocks". So seems like these are all pubkeys ever but I didn't verified that yet.
It is already huge when compressed it has over 30 GiB.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#18Nov 1, 2024, 01:13 PM
The data is live!
It is impossible for the puzzle addresses which they don't spend any satoshi. All others are already known(135-140-145....) and finding a private key (256 bits) from a public key is not realistic.
?Reply
Sign in to reply to this topic
Related topics
- Need Help! Funds Gone / Wrong Address 10
- Vanitygen: Create Your Own Bitcoin Address with This Tool 19
- Bitcoin Core displaying coins (UTXOs) in a new tab 13
- Are you in favor of BIP-110? Let's get a Bitcoin poll going. 0
- Erlay seems to have some issues here’s a better proposal for a bitcoin protocol without invites 3
- New Optional Hourglass Implementation is Live 3