Initial block download is really slow now

So, I'm trying to run a bitcoin node for the first time. I got bitcoin core from the official site. I'm using `bitcoind` on a Raspberry Pi 4 with a Samsung T7 that has about 220 GB free space (just checked a couple minutes ago). I kicked off the blockchain download on August 17, which was exactly two weeks ago. For the first week or so, it was pulling down about six months of blockchain daily at a pretty steady speed. But for the last 3-4 days, it’s really slowed down, and now I’m only getting one or two days of blockchain daily. At this pace, it looks like I’ll need a whole year to finish the initial download instead of just a few days like before. There’s no error showing, it just seems super slow now. Is this a normal thing?

Which model Pi do you have exactly? Google tells me they range from 1 to 8 GB RAM, which is the first thing I suspect to be the cause.

It has 8 GB of RAM. ``` $ free -h                total        used        free      shared  buff/cache   available Mem:           7.6Gi       452Mi        79Mi       456Ki       7.2Gi       7.2Gi Swap:          511Mi       324Mi       187Mi ```

Did you edit the bitcoin.conf file? If not, try editing it and add dbcache to 8GB of RAM. Since you have 8GB, dbcache=4000 might help speed things up. Or try this config below for Raspberry Pi.It was generated from the jlopp config generator setup for Raspberry Pi. Paste it to your Bitcoin.conf file and save, restart the node, and check for improvements.

If OP doesn't use run other app that use lots of RAM, i would recommend higher value (dbcache=6500) and enable ZRAM. Setting by that generator website would slow down IBD process, since it reduce default dbcache value from 500 to 100.

and `top` is: Also, using `iotop` I get: all the time. This could be the bottleneck.[/code]

why oh why did you use a raps pi and not an inexpensive laptop and what size is that external ssd 1tb? You are about the 20th person that has done this and asked the same questions. to all of them I say do not use a rasp pi. and use an empty 2tb ssd not a 1tb ssd lastly. 16gb ram is helpful. as an aside what interest connection do you have/ some services will punish you for doing a 600gb plus download.

I left `bitcoind` running for some hours in a very powerful machine and can confirm that now it is going very fast again, so the bottleneck is something in the Raspberry Pi. In the powerful machine I am running it with the default settings, i.e. no bitcoin.conf, except for `-datadir` which I point to the USB storage SSD. In this powerful machine, `pmap` reports a total memory usage of `bitcoind` of about 14e9. It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`.

Your chainstate directory will grow to about 12 GB, which means anything less than that largely increases disk activity.

Your first reply has a clue on this. It may say 7.2Gi "available" memory, but that's after the 7.2Gi "buff/cache" are written to your disk. Given that Core is already using the drive extensively, increasing the database cache doesn't help as much since it will otherwise require your OS to swap more memory, further increasing the disk usage. For the new setting like dbcache to apply, you'll have to restart Core (just in case you missed it instead of the above)

I am curious, if I may ask, why is it so memory hungry? From my limited understanding, while downloading the blockchain it is also validating it. This has to be done one block at a time, because of the nature of the blockchain itself (I guess I can validate a random block, but at some point I will need a hash of the previous, which I must check is valid before I can say this one is valid). If each block weights 1 MB, then how are we needing > 8 GB?

The blocks directory contains all confirmed transactions. The chainstate directory contains all knowledge on all existing UTXOs ("funds"). With each block Bitcoin Core downloads, it verifies all transactions in that block, and needs to check if no UTXO that doesn't exist gets spent (as that would mean Bitcoin is created out of thin air), and it updates chainstate by removing the spent UTXOs and adding the newly created ones. So for each block, it has to check (up to) thousands of UTXOs, and chainstate tends to grow over time. If it fits in your RAM, it's quite fast. If it doesn't come from RAM, it has to read and write thousands of UTXOs from disk for each block it verifies. Even on SSDs that just takes time, and USB only makes things worse. I'm not a blockchain expert, so this is my layman interpretation

For additional information, 1. https://statoshi.info/d/000000009/unspent-transaction-output-set?orgId=1&from=now-5y&to=now&timezone=browser&refresh=10m shows total and size UTXO growth in last 5 years. Peak UTXO size was about 12.8GB. 2. AFAIK the read/write for those UTXO considered as random read/write operation. So you'll notice the slowdown if you use HDD, very cheap SSD or other slow storage.

I'm wondering if the bottleneck issues OP experienced with the Pi 4 could be avoided by using a Pi 5? I'm asking because I'm planning on doing exactly this to set up a node and Fulcrum server. I prefer this option to a laptop because I want something portable that I can travel with. The Pi 5 has a 16GB RAM option, plus it has a faster CPU and PCIe. It seems to me these specs should make the Pi 5 a capable cheap option for running a node with a fast 2TB ssd. Any reason not to do this? I don't need the fastest sync time, don't mind waiting a few days. But wouldn't want to be the 21st person to have this issue

The RAM/USB storage bottleneck can be avoided, just make sure configure Bitcoin Core to use most of RAM capacity. But if you don't need customizability offered by Pi 5, you may want to consider Mini PC. At similar cost, you can find mini PC that use somewhat faster CPU (i refer to Intel N100 for comparison[1-2]). [1] https://www.cpu-monkey.com/en/compare_cpu-raspberry_pi_5_b_broadcom_bcm2712-vs-intel_processor_n100 [2] https://www.cpubenchmark.net/compare/6054vs5157/BCM2712-vs-Intel-N100

I'll consider the mini PC option as well, thanks, although I like the aesthetics of a Pi with Argon case. I might just stick with my clunky 15-inch Thinkpad running Debian for now--save myself the expense of any new hardware purchases, except for the 2tb ssd I'll have to buy in any case. My current 1tb ssd is running out of space with the growing blockchain, I recently had to delete a bunch of files when my node stopped working. Would be nice to replace the thinkpad with a much smaller and lighter server device, but I'm used to lugging it around on trips as a second laptop. so I can live with it for another year or two. Shouldn't be long before the Pi 6 comes out, might as well wait for that...

I didn't see anybody clarify this so here it goes. This is normal and there doesn't need to be any errors. When you download the blocks from first year or two, you are downloading blocks that were mostly empty. So your verification is basically computing a bunch of SHA256 hashes which are very fast and little to no signature verification which is the slowest part of the computation. As you move forward in the history, the bitcoin adoption grows so does the number of transactions in each block. Therefore as the blocks become fuller, you'll have perform more computation (more expensive signature verification compared to hashes) which slows your verification process down.

Thanks to mempool spammers, and it will only get worse if we don't stop it. Soon it will be impossible to run a Bitcoin node on a hardware that is not compromised (all modern hardware is). It is no longer viable to fully sync with a nice piece of hardware such as the Thinkpad x60 anymore because the CPU and RAM limitation gets a beating thanks to the anti-Bitcoin movement by Jameson Lopp and other spammers powered by Core 30.

You can up to some point replace RAM for disk, it just gets much slower. Even with an SSD, 4 GB RAM is just low. But if you insist on using dinosaur hardware, you will eventually run into it's limitations. If you're truely paranoid when it comes to running modern hardware, and also don't want an SPV wallet, you'll just have to be very patient.

You can get safe laptops with allow for 16GB ram, but it would be cool if we can get rid of spammers at some point so Bitcoin goes back to handling monetary transactions only because eventually they will only make things worse.