Copying Bitcoin blocks for faster sync, need help

19 replies 33 views
n1nja420Member
Posts: 8 · Reputation: 169
#1Dec 12, 2020, 01:19 AM
Hey everyone! Instead of grabbing the whole blockchain for my bitcoincore wallet, I thought I’d just copy it from a buddy to speed things up. This is my first time doing it like this. Could you guys help me out? What specific folders do I need to copy? I see I have two big folders in the installed wallet Blocks (with an index folder inside) and Chainstate. Do I need to copy both of these?
4 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#2Dec 12, 2020, 03:13 AM
Yes. Make sure Bitcoin Core isn't running while copying.
1 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#3Dec 12, 2020, 03:57 AM
Don't forget to check free space of your storage drive, since current size of those 2 folders is about 655GB or a bit more.
2 Reply Quote Share
QuantumYieldSenior Member
Posts: 117 · Reputation: 813
#4Dec 12, 2020, 04:34 AM
You can do it but there is risk. It's recommendable to do Initial Blockchain Download (IBD) by yourself with your own node. It takes more time but it's safer than copy the blockchain from someone else or any available source. Don't do what you're doing is safer. Bitcoin Core pruned blockchain: download it here! (DON'T DO THIS!)
3 Reply Quote Share
n1nja420Member
Posts: 8 · Reputation: 169
#5Dec 12, 2020, 09:24 AM
If you copy it yourself? Safe?
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#6Dec 12, 2020, 09:59 AM
If you trust your friend and his computer.
3 Reply Quote Share
alt21Senior Member
Posts: 398 · Reputation: 1732
#7Dec 12, 2020, 01:55 PM
Why not let it download the blockchain though? You sacrifice security for time. But the time needed to download the blockchain isn't so terrific... By the way, copying 700GB of data, how long does it take? What if the data get corrupted in the process?
6 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#8Dec 12, 2020, 02:11 PM
I would speculate OP's internet is relative slow, where visiting his friend while bringing his external HDD/SSD is faster or more convenient. And i wouldn't worry about data corruption, unless old storage drive is involved.
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#9Dec 12, 2020, 07:56 PM
I recently copied the data from a 1 TB USB 2.0 disk, which took about 6 hours. In that case, letting Bitcoin Core verify the data after copying is still a good option. But: from what I've seen, usually it's the lack of RAM that slows down Bitcoin Core's IBD, and not the internet speed. In that case verification after copying still takes about the same amount of time. Me neither. A drive could just as well fail later.
2 Reply Quote Share
alt21Senior Member
Posts: 398 · Reputation: 1732
#10Dec 12, 2020, 08:08 PM
Yes, I believe it's because the dbcache size isn't enough to exceed the actual memory size of some blocks. Would you do that though? That's the question I ask myself. I would choose to download and verify myself, but I realise there may not be a huge danger involved if done otherwise.
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#11Dec 13, 2020, 05:59 AM
Blocks are small (no more than 4 MB). Chainstate is what consumes so much RAM. I wouldn't do it, but I can think of scenarios where it's the only option, for instance if your internet has bandwidth quota. MicroSD-prices are dropping faster than the blockchain is growing. It would be fun to "sell blockchains" online, but I kinda expect countries with limited internet to have an unreliable postal service.
3 Reply Quote Share
QuantumYieldSenior Member
Posts: 117 · Reputation: 813
#12Dec 13, 2020, 09:22 AM
If you do you IBD, complete it, and copy it to another computer of yours, it's safe. In your case, you copy it from your friend, risk starts from the source of blockchain, it's from your friend. Don't trust, do IBD by yourself. Boost Your Bitcoin Node Sync With UTXO Snapshots. Lopp tested it but if you can, do IBD by yourself as if you want to run and use your Bitcoin full node, you plan to use it for a very long time so convenience is not main concern, but safety is.
2 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#13Dec 14, 2020, 10:15 AM
Talking about trust, it reminds me of UTXO snapshot feature on Bitcoin Core[1]. You only download blocks after when the UTXO snapshot created and you don't need to trust the one who create it since Bitcoin Core will verify everything. Yes, assuming it's someone who i trust and i know them have good security practice. Although i probably i would perform reindex once just in case. [1] https://blog.lopp.net/bitcoin-node-sync-with-utxo-snapshots/
0 Reply Quote Share
im_lynxHero Member
Posts: 515 · Reputation: 2161
#14Dec 14, 2020, 03:12 PM
From what I read in Lopp's blog post I don't think you're fully correct with your assumption that Bitcoin Core will "only" download blocks from the UTXO snapshot on. Or did I misunderstand you? From what I understand what Lopp wrote, didn't try it myself so far, it will first sync from UTXO snapshot's blockheight to chaintip and then act like fully synced (sync progress will first jump to progress indicator of snapshot's blockheight and then work its way up to chaintip aka progress=1.000000. Then in the background the node will start a sync from Genesis block upward to actually also verify that the loaded UTXO snapshot is indeed valid and not tampered. During this background sync the node is fully usable and like synced to chaintip with progress=1.000000. I guess you can ask the node to give you block or transaction data that it doesn't yet have downloaded in the background process. I assume you could load a wallet and spend coins from it or receive coins and showing an accurate balance. Sounds like a fun test project the next time I have spare time and need to quickly spin up a new Bitcoin Core node. I also like the fact that it should work fine for a pruned node when you're constrained by storage space and still want and need a quickly fully synced pruned node. Interpreted from
4 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#15Dec 14, 2020, 09:01 PM
Actually i wasn't being explicit enough. I actually meant with UTXO snapshot, you just need to download blocks after when the UTXO snapshot is created in order to have full node and wallet (newly created ones) functionality. And i appreciate you took time to read and understand reference blog i mentioned.
2 Reply Quote Share
alt21Senior Member
Posts: 398 · Reputation: 1732
#16Dec 14, 2020, 11:31 PM
It's indeed interesting. I 've had some questions regarding pruned nodes and how easy it is for the user to change the size of the data that's stored (link for info). I guess it should be an option if you wanted to load wallets that are created before the pruning period. The downside is that you 'd need to essentially trust data that you 'd theoretically already downloaded and verified for yourself.
0 Reply Quote Share
silentchainHero Member
Posts: 473 · Reputation: 2317
#17Dec 17, 2020, 01:29 AM
From my experience the best choice is to set dbcache in bitcoin.config as dbcache=1/4 RAM (in the case of  RAM equals to  16 Gb for instance  dbcache should be set as  dbcache=4096). Rest of RAM should be reserved for OS  operations. Also it is useful to add blocksonly=1 entry into  bitcoin.config file as this would speed up a bit IBD procedure.  IBD is not so frightful as it is painted.
4 Reply Quote Share
alt21Senior Member
Posts: 398 · Reputation: 1732
#18Dec 17, 2020, 02:56 AM
It definitely isn't frightful. But personally, I 've had best results setting dbcache=5000 on a 8GB total RAM, which corresponds to ~60% of total RAM. This is because my node is set up on a dedicated device and not on a device that requires too much free RAM for other user interaction.
1 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#19Dec 17, 2020, 07:55 AM
That depends on how you use the rest of your computer. On a dedicated node, dbcache can be much more than 25% of your RAM. This is what it does: I think this means it only keeps mempool empty, and I don't think that matters much during IBD.
0 Reply Quote Share
silentchainHero Member
Posts: 473 · Reputation: 2317
#20Dec 18, 2020, 08:04 PM
This is true. My machine is not dedicated one.  I don't agree. It really matters as it results in the faster IBD because the node doesn't have to fully validate those  transactions which are not in the blocks. As we know some nodes may keep the low-fee-trxs  for the period which exceeds that one required to sync entirely. Thus blocksonly=1 eliminates the need to validate such transactions at IBD. The gain is not big compared with the total time, but it is still a gain. Agreed.
4 Reply Quote Share

Related topics