How to scrape for Base58 address batches

Hey everyone, what's the best method to scrape the web for a ton of Base58 (legacy) addresses? I’m looking for ways to dig up as many as possible and check their transactions and balances. Appreciate any tips!

Run a full node with modifications so that it can index the blockchain based on addresses like what the block explorers do so that there is an up to date list of addresses with all UTXOs. Otherwise you can only use third party services like the existing block explorers with their limitations to get a historical snapshot of the blockchain, addresses or UTXOs. Like this one to get addresses (base58 and others) with balance alongside maybe this one.

Blockchair.com has data dumps. https://gz.blockchair.com/bitcoin/addresses/ The file for Bitcoin addresses on 22-May-2024 01:23 is 1 GB. You can download the file (.tsv) and do your filter to check Legacy addresses and balances.

Thank you Gentlemen, excellent suggestions!

OP also could run either block explorer/Electrum server, where OP could directly access database created by those. It's great suggestion. Although LoyceV website seems to be down where my browser shows ERR_CONNECTION_RESET or PR_IO_TIMEOUT_ERROR. It's not practical due to default 10 kB/s limit though.

It's not down since I can access it using my Tor browser. However I can't access it normally! The DNS lookup fails which is weird.

It works for me: Just in case, I restarted Apache2. That took a lot longer than I expected, I don't know why. Maybe it needed a good kick Try without "http://", that made it work here. For "loads" of addresses, I recommend this: Click the quote to download the files.

Thank you LoyceV, this is very useful. Is there a way or command you could recommend to sort or isolate Base58 (legacy) addresses from the TSV sheet > to a data.txt file? Thanks.

Use grep: I did it for you: legacy_and_balance.tsv (file will be deleted in 24h).

Thank you LoyceV !! this is awesome ! Ill be able to do it from now on.