So, I'm setting up a bitcoin node, using Bitcoin Core Ver 26.0. It started indexing pretty quickly, but now it's barely moving at all. Bitcoin Core crashed twice on me, once because my hard drive went to sleep after I forgot to change the setting, and I'm not sure what happened the second time. Could that have messed up the indexing? When I restarted it, it booted up fine, but the progress is just crawling. I ended the program using control q to shut it down cleanly and rebooted my PC, but no luck. I'm stuck with 361,355 blocks to go, and it says it’ll take 10 days to finish. It's been running for 30 hours already. It was down to 47 hours last night but now it’s just going down really slowly. Any ideas? Thanks!
New node indexing is really slow now
13 replies 163 views
SwiftMinerSenior Member
Posts: 259 · Reputation: 1036
#2Dec 9, 2020, 12:47 AM
What you are experiencing is normal.
Firstly you don't download the node rather the computer you are using runs as the node. You are actually downloading all of Bitcoin's block chain data and it's actually over 500Gb except you are trying to run a prune node by download some data and omitting some from a particular time frame.
Depending on the computation power of your computer and network speed, downloading the Blockchain data from the Genesis block till date and also syncing, could take a couple of days. The faster you computational power and network strength, the faster the syncing process.
Make sure you have a storage of up to 1tb preferably SSD to prevent issues due to space as it syncs and also when you attempt running it.
Thank you for the replies. I am running a PC with 8gb ram and and an i7 processor and 400 Mbps internet connection. I have an attached Samsung T7 2tb external drive to run a full node, not pruned. It just seemed like it was indexing faster and faster than all of a sudden barely moving. If it's still downloading the blockchain regardless of speed, does that mean there was no corruption in the blockchain that is downloaded so far?
When you say start resyncing, you mean delete everything and start over?
Increasing dbcache=<about half of RAM in MB> might help a bit (add in your bitcoin.conf file).
blocksonly=1 during initial blockchain download (IBD) lets your node only deal with block data over the network, don't bother at that time with unconfirmed transations.
External storage via USB isn't ideal, USB protocol eats some performance and with an UTXO set of around 16GB size your storage has to shuffle quite some data around when it doesn't fit all in your RAM during blockchain sync.
With optimized settings for a Raspi 4B with 8GB RAM and a 1TB SSD attached via USB3 a full IBD took about 4 days some months ago (my internet speed is 100MBit/s downstream).
I was kind of surprised, I expected more time as a Raspi 4B isn't really fast compared to normal PC CPUs. I did this with an Umbrel node as an experiment and I hope Umbrel doesn't use some sort of shortcuts for IBD. I might still have the debug.log of this experimental IBD.
byte_protoFull Member
Posts: 84 · Reputation: 625
#5Dec 9, 2020, 12:23 PM
I don't think you have to restart syncing, try Increasing the database cache size to since you are using an external drive this would increase the syncing process
yes but you don't have to use this
Firstly, check the usage of your hardisk if it's getting to 100% it will definitely be slow no matter what .
I've been unable to download mine due to low bandwidth in my country but I found a great trick that makes is fast .
Get an external SSD drive
Create a copy of your bitcoin data in the new SSD drive and rename the initial to another name to serve as backup incase of any error
Now follow this step as directed from @NotATether if you're using the GUI mode
In addition , the file is the bitcoin and you should save it as a conf file not as a text (note this)
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#7Dec 11, 2020, 06:32 AM
There are 2 things I would improve: get more RAM, and don't use an external drive. It's very easy to accidentally disconnect the USB cable, and that will lead to file corruption. Chainstate when fully synced is 11 GB, that doesn't fit your RAM which will lead to terabytes of data written to your disk.
Adding RAM will improve your overall computing experience, and is not very expensive anyway.
What If I use a no sdd drive?? Just asking because I got an external ssd drive because my disk functionality was been used up entirely to 100% while synchronising.
Does the RAM provides a way of not getting the disk used up ??
What is a "no SSD drive", a spinning magnetic platter aka HDD drive?
What kind of an interface does your external drive use? Most common would be USB3 or up and USB protocol, especially when UASP is not used (your USB adapter needs to support it and the driver must have it enabled, that's not always on by default, depending what OS you use), slows down the performance of a SSD quite a bit. That's what LoyceV summerized with "internal drives are preferable", especially for SSDs.
More RAM means, more data can be kept in RAM where access speed is still orders of magnitude faster than for internal and to greater extend for external drives. Less often data needs to be flushed out to storage. The OS has more space for file I/O buffers, can better optimize how data is written out to storage (moving R/W heads for HDDs is snail slow compared to flash storage with no moving parts).
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#10Dec 11, 2020, 11:14 AM
Without enough RAM, that sucks If the 11 GB from chainstate doesn't fit in your file cache, you're going to be scraping that HDD.
I have to ask: if your system still runs on an old HDD, why don't you replace it for an internal SSD instead of external? I made that switch years ago, and I'll never go back! Buy a decent one: some (budget) SSDs have low performance on sustained writes.
I did another IBD test recently, with 8 GB RAM, Linux, and SSD, and the SSD was the bottleneck. It's was reading/writing at maximum capacity for hours in a row.
Yes.
Explains why my OS crashed without giving any sign
Almost lost my important files though
Or does Windows sucks entirely??
Yes
Never cared about HDD until I was running a bitcoin Core on the PC
humbleledgerLegendary
Posts: 1027 · Reputation: 6554
#12Dec 12, 2020, 02:44 PM
Not really: the OS should kill the program that runs out of memory instead of crashing.
But I got a crashed os luckily I know how I could use the cmd-pmt for the retrieval of my file before resetting because i do keep my files hidden in the program data directory, what could have triggered it because It's was my first time experiencing it and that was when I started running the Bitcoin core
This is extremely slow. Last time i had to perform reindex, it took about 1-2 days where all files stored on HDD.
And if you don't use your PC for other task, you can close most application and background application so you can let Bitcoin Core use a bit more of RAM.
I recommend you to create new thread about your problem, rather than continue discussing on this thread.
?Reply
Sign in to reply to this topic
Related topics
- bitcoin-qt sync issues and slow speeds 6
- Got a free laptop, need help setting up Linux for a node 19
- Affordable quiet mini PC for running a Bitcoin node and blockchain explorer 19
- Recovering a Bitcoin wallet and address 8
- Creating a list of non-zero wallets and their balances using Python 4
- CipherSeed: A Comprehensive Guide to Super-Secure Crypto Wallet Backups 5