I kicked off running a node a few days ago and it was syncing quickly with around 20mbps internet. But today my internet's better, sitting at about 90mbps, so I figured it should be even faster. Instead, the verification speed per hour has actually dropped compared to when I had slower internet, which is slowing everything down. Any ideas on what could be going wrong?
What might be causing the issue
19 replies 357 views
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#2Sep 24, 2023, 05:17 PM
That's normal: as chainstate grows, syncing becomes more demanding on your hardware. Get enough RAM (16 GB is the minimum for a smooth sync) and set dbcache to a high enough value (but not all your RAM).
How do i set the dbcache to a high value?
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#4Sep 24, 2023, 08:52 PM
If you can't check the GUI Options or Google this, you should probably not be running Bitcoin Core.
Ive actually set it to 3000 its was 450 thank you.
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#6Sep 25, 2023, 02:39 AM
If you're comparing it with the earliest blocks when you just started syncing, that's because the majority of those are nearly empty blocks.
And heads-up, once its nearing the tip of the blockchain, it'll even get slower as your node starts to verify all of the scripts in the block that's skipped in the older block heights.
Currently, it's after block height 912683 (ref: /v30.2/src/kernel/chainparams.cpp#L122)
Have you considered this setting based from your free memory?
Because an improperly set value of this could even worsen your disk's performance if it makes your machine to use "swap memory".
May be you should be careful with the use of AI in finding solutions to technical issues, you think I dont know there was AI before I asked the question here ? AI could even cause you more problems because he might not understand what youre saying since it has no experience in running a node. But a lot of people here have that Thats to tell you that I believe more of the people guide here more that the AI. Most time AI could just gather garbage and give to you that why i Always ask anything that has to do with technical here and I believe that was the essence of this forum.
That should be hardware issue. You have to check the RAM and the HDD. Lower RAM can also make it slows as the downloading progresses. And if the harddisk is getting full, it will make it slows. These are some of the possible thing that can make your downloading slows. And if they are the cause, you have to upgrade the RAM, and remove some of the things in the storage to free it.
kevin_pr0toMember
Posts: 8 · Reputation: 163
#9Sep 27, 2023, 12:14 AM
Even if your internet speed is very high, the device must have the capacity to handle that speed. If your device is weak, it will not work properly. Or any other device in your house that is connected to the internet may also experience a slowdown. To sync the node dynamically, disconnect other device connections. If any other device is running, the sync will work but the speed may be slightly reduced.
coin_sigmaLegendary
Posts: 1275 · Reputation: 5553
#10Sep 27, 2023, 05:24 AM
Actually, the internet also gives you a "hard bottleneck." Even if you have good PC specs (SSD, 32 GB of RAM, and CPU with 2mbps of internet speed), expect the syncing process to be slow.
So you should check the internet speed first because that is the first requirement before upgrading your node with a better RAM and SSD.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#11Sep 27, 2023, 07:52 AM
You need a fairly empty 2tb ssd.
You need 16gb ram
You need an Intel i5 8th Gen or better.
You need a good ethernet port on your machine
A 90mb internet connect is okay but not great.
Even with all above you may not have a good node to down load from.
I have a Mac studio with 36gb ram an m4 cpu 10gb internet port a 4tb ssd empty like 200gb on it.
My tested Internet speed is 1g
But most of my download speed is 50 to 85 mostly due to the peer I connect to so if you really can do speeds like 90
You need a fast peer to connect to.
Try stopping the download wait a minute or 2 then start again
This is assuming
The cpu the ram the ssd are good
This has been explained to you, but to add: Early in the IBD, your node was downloading data, which is where the internet speed alone matters much but later on when synchronization progressed, the node began to verify blocks and transactions. At this time, the load went to the CPU. Can you mention your set up or confirm that the synch was successful.
For real?
Are you operating a data center?
Even if 1gb test speed doesn't equal 1gb in real download, I still think it's an overkill. A 100Mbps can efficiently handle the task
coin_sigmaLegendary
Posts: 1275 · Reputation: 5553
#13Sep 30, 2023, 09:28 AM
Before there was no 1 Gb internet speed, but nowadays it exists. In my rented house we have an internet speed of 700mbps,. If we downgrade to 100mbps I am sure my sons will complain about the slow internet and ping spikes.
However, 100mbps should be enough to download blocks and necessary files when syncing; it's not a bottleneck even I think 50mbps is enough.
If 50mbps didn't help you speed up the syncing process, then the issue is not the internet; it's your hardware specs, but having an SSD is a huge improvement.
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#14Oct 2, 2023, 02:38 PM
50usd for 300mbs
65usd for 1gb
You are correct i do not need the 1gb.
And the only way it speeds up my node download is if I connect to a node with 1gb speed.
But I do a lot of Internet in the home so I pay the extra for that speed.
Im currently running a node with a 1tb ssd system and its pretty fast enough
A system with a ram of 8gb should be equal to the task 16gb will be better but I think its over stretched.
Absolutely true, this specification should be the minimum for a fast setup
True if the nodes youre peering with are not very fast in relaying information/ data to your node you might actually experience slower progress therefore for it is very true that the peers youre connected to are also a factor that determines the speed of your download.
I do that always and it worked for me Infact Im running a full node currently and I can tell its what I do when its lagging.
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#16Oct 3, 2023, 05:56 PM
You must be talking about the system's performance, in which case, can be true in most OS.
But for a Bitcoin Node, 8GB isn't as good as 16GB.
Since the current discussion is about "to sync as fast as it can", it's more efficient if the whole UTXO-set can be stored in the cache which isn't possible with 8GB of RAM.
That way, it wont have to partake to one of the biggest bottleneck which is disk I/O.
Although if the dbcache config isn't touched or set to a low value, you may not see the benefit of having a higher RAM.
E.g.: Two systems (8 & 16 GB RAM) with dbcache of 4GB or set to default, the performance may be similar if there are no other running programs.
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#17Oct 3, 2023, 09:56 PM
You know online games worked through telephone modems too, right? It may be an unpopular opinion, but I think high internet speeds are overrated.
Even at 10 Mbit/s, it only takes a week to fully sync Bitcoin Core from scratch. The only limitation is countries where bandwidth is limited.
You pay for the hype
We're at 70 Mbit/s, and 99% of the time that's not a limitation. I just checked: each "step" faster costs only a few euro per month extra. That's because most people don't use more bandwidth if their internet speed gets 10 times faster.
With 8GB RAM, my system wrote many TB of data to my disk during sync. My SSD (not NVMe) was the bottleneck last time I tried.
I highly doubt that. I expect Linux standard disk cache to fully use that extra 8 GB RAM. I'd test it, but I kinda don't want to write an additional 10+ TB to my spare laptop with 8 GB RAM and small SSD. That severly limits its lifespan.
Someone already test it on 32GB RAM system, https://blog.lopp.net/effects-dbcache-size-bitcoin-node-sync-speed/. In short, OS caching work great. But setting higher dbcache allow somewhat faster sync speed.
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#19Oct 4, 2023, 06:02 AM
That's what I'm talking about, it'll have to dump data from the UTXO set to your disk more frequently which will compete with the gigabytes of data that's already being written.
Whereas having a RAM capable of utilizing higher dbcache value to store the whole UTXO-set, it'll be significantly reduce disk writes.
It may still perform reads though, specially if he restarts his node often.
That is just a rough comparison in the node's performance if the user didn't set an optimal dbcache (set to default or too low).
The point is, if the node doesn't utilize the free memory effectively, it's not benefiting much from higher RAM.
Setting it too high isn't good either as I've mentioned in my first reply.
I think I understand your point but I want to ask if that also applies to a pruned node
Related topics
- Issue with Bitcoin Core Wallet after power outage 8
- mandatory-script-verify-flag-failed (Non-canonical DER signature issue) 19
- bitcoin-qt sync issues and slow speeds 6
- Recovering a Bitcoin wallet and address 8
- Creating a list of non-zero wallets and their balances using Python 4
- CipherSeed: A Comprehensive Guide to Super-Secure Crypto Wallet Backups 5