hey everyone. i messed up and deleted my blocks and chainstate folders. i was using a pruned blockchain setup, and i thought when I rebooted the server, it wouldn’t need to download the whole blockchain again since i set it to prune=2000. but it seems like it’s pulling the entire chain again, so i stopped it for now. luckily, i have another copy i can use to replace what i need.
before i do that, though, i just want to confirm if i really need to copy over the whole chain before proceeding with the pruning. is there maybe some other way to handle this?? thanks!
accidentally wiped my blocks and chainstate folders
18 replies 207 views
SwiftMinerSenior Member
Posts: 259 · Reputation: 1036
#2Oct 27, 2022, 12:51 PM
Pruning actually relies on a full download Initially Pruning doesn't mean it downloads only the last 2000 blocks. Instead, it downloads the entire chain and then discards older blocks according to your pruning setting. This is because the software needs to verify the entire history to ensure the integrity of the current state.
Now yours didn't work out because the deleted directories contain crucial data like the block index, chain state, and block data. Without them, the software can't reconstruct the chain's history or verify new blocks. The node actually needs to download all of the blocks in order to rebuild the index of the chain.
thanks for the reply
FYI, prune=2000 actually means asking Bitcoin Core to store most recent blocks, up to 2000MB in size. In practice, it could be more or less than 2000 blocks depending on recent block size.
silentchainHero Member
Posts: 473 · Reputation: 2317
#5Oct 28, 2022, 04:05 AM
The good things about this is that Bitcoin Core does it on the fly so even during IBD the required storage space can remain relatively small.
That said, I personally prefer running a full node with the copy of entire blockchain rather than a pruned one. Having the full blockchain gives me the flexibility to rescan the blockchain if I ever need to recover a wallet, which isnt possible with a pruned setup.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#6Oct 28, 2022, 07:30 AM
yeah pc's good enough to do a full node are cheap
as are laptops.
So many used ones on eBay .
https://www.ebay.com/itm/234989817127?
this is 200 buy a decent ssd
https://www.amazon.com/SAMSUNG-Technology-Intelligent-Turbowrite-MZ-V9S2T0B/dp/B0DHLCRF91/ref=sr_1_5?
cost 339 for the 2 and you are set for at least 3 to 5 years.
that was a fast search you can likely find a good setup for under 339
here is a decent laptop for 359 this is plug n play
https://www.ebay.com/itm/396388013020?
these are fast eBay searches if you take your time you can find a better deal
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#7Oct 28, 2022, 08:12 AM
The last time I did a rescan on my old laptop with block on HDD, it took a very long time. Given enough RAM, bandwidth and CPU power, even a pruned node can do a rescan in just half a day. But I admit: if you have that kind of hardware, you probably have enough space to store a full blockchain too.
SwiftMinerSenior Member
Posts: 259 · Reputation: 1036
#8Oct 29, 2022, 11:16 AM
To be honest PC specs that we use to regard as huge or high level specs back in the day are now regular specs because of the demand for better GPU CPU and even RAM on laptops and other PCs. Nevertheless on e thing I've noticed is that a storage has a role to play in speed though especially if you make use of SSD.
That's the reason most people prefer to go for a prunned node since you have more storage to spare hence the risks of lagging and glitches are way low.
i have a full node running on my rapberry pi with a 1TB m.2 ssd. when i bought the ssd 3 years ago i think the chain was 450GB(?) about. now the storage is up to 700GB. I'll have to set my sights on a 2TB soon
I have this pruned node on my desktop which I daily drive so I can experiment with different settings and not have to constantly reboot the server on the raspberry pi.
I took the 1TB storage from the raspberry pi and mounted it to my pc to copy the blocks and chainstate files. still took a few hours but better than waiting for a full IBD.
I cant imagine having to do a full IBD in a few years time when the blockchain is over 1TB.
maybe they'll be a need to start sharing pruned blockchains with people who cant or dont want to go through an IBD.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#10Oct 30, 2022, 01:50 PM
That's not going to be a problem. Computing power is still growing at exponential rate, while Bitcoin's blockchain is only growing linear.
See Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)
so what's the approximate time now for an IDB to complete these days?
and i think it will be a problem as the blockchain grows. if you always have to download the entire chain first before pruning, it's going to be prohibitive for more and more people to run a node.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#12Oct 30, 2022, 03:11 PM
If you have fast internet six hours or less..
1gb eth speed
2tb ssd
16gb ram
A decent cpu
Now I showed links for $350 PCS THAT WILL DO FAST DOWNLOADS IF YOU HAVE 1GB INTERNET.
The issue will not be bad if the net move to 10gb speed in 5 years.
Some places have 10gb.
My newest pc will be a 10gb internet 2tb ssd 36gb ram pc i will do a download in 4 hours. As i have 1gb Internet.
looks like i'll have to upgrade to 1gb at least. i just have 300mb right now
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#14Oct 30, 2022, 10:21 PM
Depending on system specs, it's anywhere from hours to weeks.
That doesn't make any sense. It's 2 days since you opened this topic, a 300 Mbit/s connection could have downloaded the full Bitcoin blockchain 6 times already.
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#15Oct 30, 2022, 11:54 PM
Now that you've mentioned it, there'll be an "assumeutxo" feature that will enable you to use your node and wallet in just a few minutes or hours even if the background sync isn't finished yet.
You can read this for more information: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
Here's a link to follow its implementation to the reference client: https://github.com/orgs/bitcoin/projects/7
Some people would find it challenging. But FWIW,
1. Time and cost to perform IBD on Bitcoin full node is much lighter than majority of altcoins.
2. Computer hardware and internet are expected continue to grow more faster.
In this case, those people probably better using SPV/lightweight wallet.
FWIW, one of Jameson Lopp blog mentioned it took 452 minutes to perform IDB with his fast PC and 28GB dbcache. See https://blog.lopp.net/effects-dbcache-size-bitcoin-node-sync-speed/.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#17Oct 31, 2022, 07:10 AM
It is a nice article. I have found that 4-6 hours is possible with a quality machine. My latest download was
an M4 Mac mini with 16gb ram and a 2tb ssd. done on 1gb speed internet on feb 7 2025.
It took 6 hours or about 420 minutes. using the 450mh cache. I think the reason why I could be so fast with the stock cache was the ram in the 2025 m4 mini and the cpu is very likely faster than his 6 year old machine.
My Mac Studio is coming on the 16th of April
it has a m4 max cpu
it has 36gb ram
it has a 2tb nvme.2 ssd
it has 10gb internet but I only have 1 gb internet.
I will do a 28.1 core download with it and set an 18gb cache . I think I can do a 4 hour download.
in theory I could do this in under 2 hours. but I do not think it will
see calculator below
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#18Oct 31, 2022, 08:14 AM
I'd like to test this on a high power server, but the only ones I can rent by the hour (with my existing crypto-accepting hosting accounts) aren't really high-end.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#19Nov 1, 2022, 06:51 PM
The mac studio should do a really fast job. I will post results on Friday.
I HAD A thread ripper with 128gb it was really fast.
That was in 2021. But my internet was slower only 200mb.
I would love to get a 10gb internet line but I am in suburbs outside new york city I don't know of any internet service faster than 2gb within 20miles of me.
I could upgrade to 2gb for 20 more a month.
And a 200 dollar rebate which actually makes sense once I get the mac studio. Since it has a 10gb connection