Hey everyone! Instead of grabbing the whole blockchain for my bitcoincore wallet, I thought I’d just copy it from a buddy to speed things up. This is my first time doing it like this.
Could you guys help me out? What specific folders do I need to copy? I see I have two big folders in the installed wallet Blocks (with an index folder inside) and Chainstate. Do I need to copy both of these?
Copying Bitcoin blocks for faster sync, need help
19 replies 33 views
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#2Dec 12, 2020, 03:13 AM
Yes.
Make sure Bitcoin Core isn't running while copying.
Don't forget to check free space of your storage drive, since current size of those 2 folders is about 655GB or a bit more.
QuantumYieldSenior Member
Posts: 117 · Reputation: 813
#4Dec 12, 2020, 04:34 AM
You can do it but there is risk.
It's recommendable to do Initial Blockchain Download (IBD) by yourself with your own node. It takes more time but it's safer than copy the blockchain from someone else or any available source.
Don't do what you're doing is safer.
Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)
If you copy it yourself? Safe?
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#6Dec 12, 2020, 09:59 AM
If you trust your friend and his computer.
Why not let it download the blockchain though? You sacrifice security for time. But the time needed to download the blockchain isn't so terrific...
By the way, copying 700GB of data, how long does it take? What if the data get corrupted in the process?
I would speculate OP's internet is relative slow, where visiting his friend while bringing his external HDD/SSD is faster or more convenient. And i wouldn't worry about data corruption, unless old storage drive is involved.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#9Dec 12, 2020, 07:56 PM
I recently copied the data from a 1 TB USB 2.0 disk, which took about 6 hours.
In that case, letting Bitcoin Core verify the data after copying is still a good option. But: from what I've seen, usually it's the lack of RAM that slows down Bitcoin Core's IBD, and not the internet speed. In that case verification after copying still takes about the same amount of time.
Me neither. A drive could just as well fail later.
Yes, I believe it's because the dbcache size isn't enough to exceed the actual memory size of some blocks.
Would you do that though? That's the question I ask myself. I would choose to download and verify myself, but I realise there may not be a huge danger involved if done otherwise.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#11Dec 13, 2020, 05:59 AM
Blocks are small (no more than 4 MB). Chainstate is what consumes so much RAM.
I wouldn't do it, but I can think of scenarios where it's the only option, for instance if your internet has bandwidth quota.
MicroSD-prices are dropping faster than the blockchain is growing. It would be fun to "sell blockchains" online, but I kinda expect countries with limited internet to have an unreliable postal service.
QuantumYieldSenior Member
Posts: 117 · Reputation: 813
#12Dec 13, 2020, 09:22 AM
If you do you IBD, complete it, and copy it to another computer of yours, it's safe.
In your case, you copy it from your friend, risk starts from the source of blockchain, it's from your friend.
Don't trust, do IBD by yourself.
Boost Your Bitcoin Node Sync With UTXO Snapshots. Lopp tested it but if you can, do IBD by yourself as if you want to run and use your Bitcoin full node, you plan to use it for a very long time so convenience is not main concern, but safety is.
Talking about trust, it reminds me of UTXO snapshot feature on Bitcoin Core[1]. You only download blocks after when the UTXO snapshot created and you don't need to trust the one who create it since Bitcoin Core will verify everything.
Yes, assuming it's someone who i trust and i know them have good security practice. Although i probably i would perform reindex once just in case.
[1] https://blog.lopp.net/bitcoin-node-sync-with-utxo-snapshots/
From what I read in Lopp's blog post I don't think you're fully correct with your assumption that Bitcoin Core will "only" download blocks from the UTXO snapshot on. Or did I misunderstand you?
From what I understand what Lopp wrote, didn't try it myself so far, it will first sync from UTXO snapshot's blockheight to chaintip and then act like fully synced (sync progress will first jump to progress indicator of snapshot's blockheight and then work its way up to chaintip aka progress=1.000000.
Then in the background the node will start a sync from Genesis block upward to actually also verify that the loaded UTXO snapshot is indeed valid and not tampered. During this background sync the node is fully usable and like synced to chaintip with progress=1.000000. I guess you can ask the node to give you block or transaction data that it doesn't yet have downloaded in the background process. I assume you could load a wallet and spend coins from it or receive coins and showing an accurate balance.
Sounds like a fun test project the next time I have spare time and need to quickly spin up a new Bitcoin Core node. I also like the fact that it should work fine for a pruned node when you're constrained by storage space and still want and need a quickly fully synced pruned node.
Interpreted from
Actually i wasn't being explicit enough. I actually meant with UTXO snapshot, you just need to download blocks after when the UTXO snapshot is created in order to have full node and wallet (newly created ones) functionality. And i appreciate you took time to read and understand reference blog i mentioned.
It's indeed interesting.
I 've had some questions regarding pruned nodes and how easy it is for the user to change the size of the data that's stored (link for info).
I guess it should be an option if you wanted to load wallets that are created before the pruning period.
The downside is that you 'd need to essentially trust data that you 'd theoretically already downloaded and verified for yourself.
silentchainHero Member
Posts: 473 · Reputation: 2317
#17Dec 17, 2020, 01:29 AM
From my experience the best choice is to set dbcache in bitcoin.config as dbcache=1/4 RAM (in the case of RAM equals to 16 Gb for instance dbcache should be set as dbcache=4096). Rest of RAM should be reserved for OS operations. Also it is useful to add blocksonly=1 entry into bitcoin.config file as this would speed up a bit IBD procedure.
IBD is not so frightful as it is painted.
It definitely isn't frightful.
But personally, I 've had best results setting dbcache=5000 on a 8GB total RAM, which corresponds to ~60% of total RAM.
This is because my node is set up on a dedicated device and not on a device that requires too much free RAM for other user interaction.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#19Dec 17, 2020, 07:55 AM
That depends on how you use the rest of your computer. On a dedicated node, dbcache can be much more than 25% of your RAM.
This is what it does:
I think this means it only keeps mempool empty, and I don't think that matters much during IBD.
silentchainHero Member
Posts: 473 · Reputation: 2317
#20Dec 18, 2020, 08:04 PM
This is true. My machine is not dedicated one.
I don't agree.
It really matters as it results in the faster IBD because the node doesn't have to fully validate those transactions which are not in the blocks. As we know some nodes may keep the low-fee-trxs for the period which exceeds that one required to sync entirely. Thus blocksonly=1 eliminates the need to validate such transactions at IBD. The gain is not big compared with the total time, but it is still a gain.
Agreed.