High CPU temperatures during node synchronization and checksum issues

17 replies 437 views
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#1May 17, 2023, 04:33 PM
I'm setting up a Bitcoin Knots node from scratch on Debian, and I've already run into a couple of errors related to this. As the blocks were downloading, I noticed an error pop up. This happened again after a few hours. Thankfully, I managed to keep syncing after resetting the OS, and I didn't have to reindex everything or, heaven forbid, redownload the entire blockchain, which would be a nightmare. So now the sync is running smoothly. I'm about 53.45% done, and according to the GUI, there are about 18 hours left, but that probably changes with more block activity. The issue is I'm using an i5 laptop, and the CPU cores are constantly hitting 95ºC/203ºF. The sensors indicate that anything above 100ºC is critical. I'm curious if this overheating is what’s causing the checksum errors. Is there any way to reduce the CPU usage from Bitcoin so I can keep the temperature down? I'm using a dbcache of 2048, which seems reasonable given my 8GB of RAM, and I've already set par to 1. Another thing to think about is that this node runs from a portable USB SSD, so I wonder if the USB I/O performance might not be optimal. But I really wanted the convenience of having a portable node while traveling, and I figured this would be better than using a regular USB flash drive. The disk is also encrypted with FDE. Lastly, I haven't tested the RAM with memtest yet. So yeah, something's triggering these errors. I hope I can finish syncing without any more issues after changing the par setting, but who knows. At the very least, I want to get the temperature down to around 75ºC/167ºF because this CPU is really pushing it.
4 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#2May 17, 2023, 08:17 PM
How about the hardware? When did you last cleaned your laptop's internals? That overheating is a striking hardware issue probably with your laptop's cooling since Bitcoin Core already set not to utilize it to its limit. With throttling and proper cooling, it shouldn't reach critical levels so check your laptop's heatsink, fan and thermal paste for dust buildups. If if's clean and the thermal paste isn't crumbly yet... that may be a symptom of failing CPU if it's not at 100% utilization. Or course, check your CPU's spreadsheet for its normal/max temp since there are a lot of different generations of i5.
4 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#3May 18, 2023, 01:17 AM
If your laptop only used to sync Bitcoin Core, i'm sure you can use higher dbcache value since Debian doesn't use that much RAM to improve sync speed. I don't see how RAM can make CPU overheat. But you may want to memtest it since you mentioned Bitcoin Knots exited itself. In addition, i would recommend OP to research about how to perform CPU undervolt. It's another way to reduce CPU temperature.
0 Reply Quote Share
colddiamondHero Member
Posts: 623 · Reputation: 2467
#4May 19, 2023, 03:02 PM
Like over a dozen of them. @takuma sato  what are the full specs of the laptop. An early 2010 i5 is a lot different then a 12th gen Same with the drive is it a new ssd is going to run a lot cooler then a spinning drive or even an old SSD. -Dave
2 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#5May 19, 2023, 09:01 PM
I have run the node syncing all night successfuly by using -par=1. So it looks like limiting it to 1 core did the trick. The progress speed didn't seem to be affected that much. However when I hit around 70% it has become incredibly slow. It's pushing 0.12% an hour. At this rate it will take 8 days. This is like a year ago remaining of blocks. The net activity graph shows huge gaps between the green bars. Hopefully this is not the result of schnorr signatures and then weird usage of the blockchain with more complex signatures that take more time. Because that looks like that (gaps between activity). In any case, something started happening about a year of blocks remaining where it has become so damn slow. I've tried restarting a few times to swap peers and no difference. Any ways around this?
3 Reply Quote Share
hodler2019Legendary
Posts: 2182 · Reputation: 12913
#6May 21, 2023, 02:18 PM
yeah tell us how big is the external ssd. ie 1tb is not the best choice 2tb would be better. tells us what gen is the i5 if the cpu is 8th gen or better it should be good enough are you using wifi to load or eth cable?
2 Reply Quote Share
gr3g.0rbitHero Member
Posts: 1025 · Reputation: 2646
#7May 21, 2023, 07:30 PM
Maybe not, your node probably caught up to the block height where your node starts validating all of the scripts compared to the lighter verifications done in the older blocks. I'm talking about the default "assumevalid" block value. If you're using v29.0, it's block height 886157 onwards: github.com/bitcoin/bitcoin/blob/v29.0/src/kernel/chainparams.cpp#L121 Since it's almost done syncing, I suggest not to touch that default value.
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#8May 21, 2023, 10:16 PM
Instead of cpulimit, I'd look at reducing the CPU frequency. Read about scaling_governor powersave and cpufrequtils (but I've never tested it for this purpose). When I run a laptop long term, I clean the fan from dust a few times per year. The only way to solve this: get more RAM. My chainstate is 11 GB, your RAM is 8. Even with 16 GB it heavily relies on it's SSD to sync. Look up the TDP for your laptop CPU. I never buy them >15W, but many laptop CPUs burn 35 to 45W (or even more). All this power is turned into heat. Intel built CPUs that burn a maximum of 6W and don't even need a fan, but not many laptops use them.
1 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#9May 22, 2023, 03:01 AM
This is not normal. At current speed it will take now like 7 weeks. The speed just gets worse. Something going on here. I may have to give up on the laptop and try on my desktop by plugging the USB and hopefully it works (since Debian was installed on the laptop and it would be a different hardware) It's a Sandy Bridge, it's enough, and it's eth cable plugged to a fast connection and 2 TB SSD. There's something going on that is making it insanely slow. It should be faster even on an older computer.
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#10May 22, 2023, 06:45 AM
Actually, it is normal when there's a bottleneck. Find the bottleneck, and it should improve (until the next bottleneck pops up). What disk is in the laptop? If it's SSD: symlink chainstate to your local disk.
3 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#11May 22, 2023, 06:58 AM
Yeah, it's far too slow. I would guess it's caused either by 1. Very low dbcache value (only 2GB), while peak UTXO size was at 12.8GB on late 2024[1]. 2. If you use your 2TB SSD to store other data, it's worth to know that some SSD become noticeably slower when it's getting full. 3. Too much data have written on your SSD and it switched from pSLC to either TLC/QLC mode which makes it slower. Below image example is for sequential write, but i expect random I/O activity would be severally impacted as well. But if you want to find the exact bottleneck/issue, you probably should start by using tool such as htop or iotop. [1] https://statoshi.info/d/000000009/unspent-transaction-output-set.
2 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#12May 23, 2023, 10:19 PM
The entire OS + node is installed within the same external SSD. I don't want anything leaking outside of the encrypted drive. The idea was to be able to have a full node synced on the go and carry it with you if you travel. The SSD is dedicated only to running the node, so considering the blockchain is around 685MB,  and there is only the Debian OS installed, it should last until I need to upgrade. Thankfully, it seems that after turning off the laptop for some hours and trying again, it's not at 2% per hour, which means at this rate it may finish within 10 hours, assuming it doesn't stop downloading again and huge gaps happen. So maybe the high temperature was making a problem or something. But it fluctuates, now it says 13 hours to go and 1.54%... but as long as it's not 7 weeks then I will consider this as expected performance. Edit: oh well now it's going slower again at 0.88%, 24 hours to go adnd it keeps getting slower.. hopefully it picks up speed again. Like I said above, the problem seems to have been fixed by turning off the laptop for some hours and trying again. So perhaps it was the high temps or who knows. Now temps are around 65 to below 75, which is manageable given the high summer temperature. From what i've read, as far as dbcache goes you should use 1/3 of your ram. In any case, this shows that we need Knots to become the #1 node and we need to stop the spam. We don't need all this non-monetary crap making the initial blockchain download an even bigger nightmare that scares away potential nodes that could be online from people that have laptops that they don't use lying around that could be used as nodes. It's clear now they are ruining Bitcoin with this nonsense being hosted on the blockchain.
4 Reply Quote Share
byte_orbitFull Member
Posts: 186 · Reputation: 738
#13May 24, 2023, 12:38 AM
Others have answered your thread pretty well, I would like to add something. While I do understand the use case, I would never recommend this. Putting the issue of speed aside, USB is not a reliable interface for something that is so intensive such as Bitcoin Core, neither the hardware nor software. All it takes is a small intermittent fault, a halt in the software or a twitch of the cable and you can have a big corruption. I've done some experiments in the past and I could never have a reliable node on a USB drive. Neither in the earlier days on a HDD, nor these days on an SSD. The chance for something to go wrong is just way too high compared to internal storage. This doesn't mean that you can't still try, just beware that issues are expected. Have you ever considered upgrading the laptop storage directly instead?
2 Reply Quote Share
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#14May 26, 2023, 03:02 AM
Are you saying you'll run your own OS on external USB on someone else's computer? Why would you trust them, and why would they trust you? I'd never let anyone do that on my hardware! Why not also bring a laptop on your travels? I paid just over €100 for my disposable second hand travel laptop, including SSD upgrade. It's quiet, small, light, tough and ugly (that last one is a deliberate feature). The biggest drawback is the 8 GB RAM that can't be upgraded. I still wouldn't recommend it for a full node though: if you use it once a week and it takes an hour to catch up syncing while slowing down your laptop to the point you can't do anything else, you can't really use it to make a quick payment.
4 Reply Quote Share
hash_bossLegendary
Posts: 1166 · Reputation: 5261
#15May 26, 2023, 06:26 AM
People who recommend that probably assume you also use your computer/laptop for other tasks. If you only sync Bitcoin Core and doesn't run other app that use lots of RAM, you can set it higher based on portion of free/available RAM. I agree. Some privacy can be preserved if you use SPV wallet that connect to your node or use wallet that use BIP 157/158 (such as Wasabi and Ginger wallet).
0 Reply Quote Share
im_lynxHero Member
Posts: 515 · Reputation: 2161
#16May 26, 2023, 11:35 AM
From first-hand experience I've been running a Raspiblitz and Umbrel node on two Raspi 4B with 8GiB RAM for years rock-solid reliably where the main blockchain data storage was a SATA SSD connected via an USB3-SATA-adapter to the Raspi. Never had any issues with the storage (of course the SSD (SanDisk SSD Plus) was a low-power model suitable for the limited total USB power budget of the Raspi 4B). It doesn't contradict your statement. Anyway, I'm aware that a Raspi 4B isn't comparable to a laptop or desktop computer. If possible, I'd prefer to use something else than USB for reliable 24/7/365 storage attachment and use. In June 2023 I did an experiment how fast Umbrel could sync the blockchain from scratch with a dbcache value near ~4800 for one of my above mentioned Raspis: it finished after 95h at blockheight 796033. And a Raspi 4B isn't known to have rocket fast access with such an USB3 adapter to a rather normal SATA 2.5in SSD.
4 Reply Quote Share
HumbleKingFull Member
Posts: 38 · Reputation: 340
#17May 26, 2023, 02:59 PM
If the RAM is faulty and the CPU needs to keep refreshing caches, it would affect it but in modern systems with ECC, the issue is usually caught before the CPU becomes involved.  Another thing for the OP to look at would be his BIOS - disabling certain features could cause his CPU extra work.
3 Reply Quote Share
DarkSeedSenior Member
Posts: 209 · Reputation: 1423
#18May 26, 2023, 06:21 PM
I don't see this being an issue with 3.0+ interfaces. I managed to finish syncing the node with the desktop and once signed thankfully the laptop is able to keep up. I like the portability of a small drive. Gives me more options. No, you can own diffferent laptops on different spots and avoid carrying a laptop crossing borders with important information which is not a great idea.
4 Reply Quote Share

Related topics