I wanted to kick off a general discussion about Umbrel, which is an OS that lets you run your own Bitcoin and Lightning Network node. It's super easy to get going on a Raspberry Pi, and then you can enjoy having your own node. Your node operates through Tor (which seems to be a must) and you can also manage it over WAN, not just LAN, by using your onion URL from anywhere.
There are other similar systems out there like myNode, RaspiBolt, and RaspiBlitz.
Right now, I'm dealing with a syncing issue. It’s taking way longer than I thought it would. I set it up yesterday, and after 24 hours, I'm only at 14.48%. Is this normal, or is the block verification taking too long? Is there any way I could speed things up? Like, can I verify the remaining blocks on my PC, which should be quicker than the Raspberry, and then just plug my external drive back into it when it's done?
My setup:
1 TB external SSD (is this enough for Bitcoin Core and Electrum?)
Internet speed: ~100 Mbits/sec.
32GB micro SD card.
It's normal. Raspberry Pi is notoriously slow and the record for me was 5 days with a fairly fast SSD and a RPi 4, can take up to weeks depending on which RPi you have. The CPU, the RAM and USB (again depending on your Pi) are bottlenecks.
That's what I did with my subsequent synchronization after it got corrupted a few times. Either that or just copy the entire dir of a copy that you've already fully synchronized (if you have).
I have RPi 4 so it'll probably take a century less.
Indeed. In order to test all those OSes you'll have to wait for a month, although I may stay with Umbrel as it seems better than myNode (that I had tried). The syncing didn't take that much on that OS, though. It finished in ~2.5 days. Does this have to do with the mandate usage of Tor? (without Tor it should sync faster)
It got corrupted? Why? Don't I just plug my SSD to the PC's usb port, start the daemon using bitcoind --datadir=<...> and safely shut it down with bitcoin-cli --datadir=<...> stop?
Probably not. The only way to check is by looking at the resource usage, htop should show you how much ram is getting utilized and the CPU as well. I've never really had a bottleneck while using Tor, or at least it isn't that obvious to me at all.
As in my own RPi got corrupted due to an unclean shutdown. It has nothing to do with me synchronizing using my computer.
Yes. I rebuilt the block data by plugging the SSD in my computer and treating it as an external storage. It would be faster to simply copy another uncorrupted block dir onto the SSD though.
I set up a RPi node with a hard disk (not SSD) and TOR, and it took about a week to do the initial block download.
I have a UPS because power outages are common where I live.
As mentioned... IBD on a Pi can be torturously slow...
If your PC is Linux based and/or can read EXT4 formatted drives... then yes. From memory I was actually able to do this using Ubuntu running under "WSL" (Windows Subsystem for Linux)... the trick is to remember to modify the directory and file permissions after you copy them to make sure Umbrel can read them. User/Group 1000:1000 from memory.
If your PC can't read EXT4 formatted drives, you can use ssh/scp and copy the blockchain data from your PC to your external drive connected to the Pi (I have also successfully transferred the blocks this way).
You'd probably need to delete the corrupted block and then do a reindex... that should work.
Is there any way I can verify the blocks with my GPU instead? Both RPi's and my PC's CPUs are very slow and it'll take ages to finish. At the moment it's 18.62%. Also, in Umbrel I don't have many connections with other (possibly) Tor nodes which means that I may also have a slow downloading, too. Right now, I'm connected with one Bitcoin node and it sometimes, becomes, zero.
Hmm, that means around 14% per day. So I'm in a much worse situation. I have to do something (maybe using another Bitcoin node OS?) otherwise, it may take more than two months.
Does reindexing mean re-verifying each block?
Essentially, yes... The short answer is that it starts at the beginning and verifies the block data stored on disk. If an invalid block/file is encountered, it will start syncing (downloading blocks from peers) from that point to ensure the validity of all the blocks.
I'm not sure if the .onion servers are included in peer discovery or if you have to explicitly set them. The Electrum docs regarding Tor seems to indicate that you have to either set up as connecting to 1 server (ie. .onion only) or you run the risk of actually connecting to clearnet peers for blockheader download:
However, when I look through the list of "known" servers... I so see a couple of .onion servers listed:
So, it's possible that .onion nodes are included in peer discovery.
Note: Setting Electrum to use my Umbrel node (from a remote location)... it is connecting to my .onion address, but the other connected nodes are all clearnet.
So, I deleted Umbrel, burnt an Ubuntu OS to the SD card, synced with Bitcoin Core until ~450,000 and then, due to some issues, moved back to Umbrel. I forgot to add txindex=1, so I had to reindex it. I haven't done so far, because it requires much time. I copied, though, all my block files (directories: blocks, chainstate, indexes) to the Umbrel's bitcoin directory and I wanted from Umbrel to index them for me.
While it does read it that I'm using 151 GBs out of 1 TB, I don't understand if it resyncs or just re-indexes.
That 0.13% scares me.
Umbrel runs electrs so I guess it should allow others to connect with my .onion url.
Have you tried connecting to the Pi using ssh/terminal and checking what the Bitcoin Core logs say?
If you make sure you're in /home/umbrel/umbrel and then use:
That'll show you the last 100 lines of the debug.log file and then continue to output new lines (ctrl+c to stop). That should hopefully give you some idea of what it is actually doing.
Alternatively you should be able to check the network usage of the device to see if it is actually downloading blocks.
i now also have an umbrel full node running
https://twitter.com/cryptocygan/status/1492930549861826563
the blockchain is now completely synchronized but somehow my node does not get more than 11 inbound connections (peers) ... port 8333 is enabled in my router.
or is it just the outbound connections that umbrel shows me?
Mine is smaller:
https://bitcointalk.org/index.php?topic=5364113
Is that a RPi in there? I am on mobile and I can't tell if it's that or something else.
I do like Umbrel, but as I have pointed out the 'app store' thing allowing it to do much more then BTC makes me worry about the security of it.
We know what the security risks are for a lot of the crypto apps, but what if you installed Pi-Hole and someone found and exploited something with that....
-Dave
yes, there are even 2 rpi 4 built in and installed
but again back to my actual question ... how can i see how many inbound connections my umbrel node has?
the app thing is such a question i share with you - actually you take your own risk there if you would install additional apps here...
and then immediately another question about the apps - has anyone here already installed any apps on the umbrel node and can recommend which ones here?
I don't use Umbrel but looking at their code[1], the number on the dashboard shows the total connection of your node received(inbound+outbound). So you have to either check on some explorer(BTC RPC explorer) or directly SSH'ing into the server. Anyway, isn't Umbrel default network are using Tor?
You can try SSH into the server.
Use Powershell if you are on Windows or if you are running on macOS/Linux you can directly type below command on the terminal.
Then enter your password.
After you've connected to the node, then proceed to see the network info by typing this on the terminal:
And then, the output should show you how many connections you got.
e.g.