Why transaction propagation is slow without weak blocks

A great illustration of this issue can be seen on testnet3, especially when you check out the mempool status on mempool.space and compare it with your own node. Picture this: you fire up a new full node and link it to the network. When you look at mempool.space, you might think that once your node completes the Initial Blockchain Download, it’ll start pulling in those 787 MB of transactions. But that’s not what actually happens. Instead, if you jump straight into mining and start creating new block templates, you’ll notice that your blocks are pretty much empty. This raises a question: why is that? Well, the answer is pretty straightforward. Transactions are sent out one at a time, so if you start mining right away, your blocks won’t have many transactions in them. This also means that other miners using Bitcoin Core with the default settings are experiencing the same thing. So if you’re checking out mempool.space and following their suggested fee rates, you might end up overpaying. An almost empty block doesn’t necessarily mean there’s some sort of censorship going on. If the block health looks low, it could just mean a new miner joined the scene, and that node is fetching transactions one-by-one. It’s going to take a while for it to catch up to what mempool.space is showing right now. In the meantime, that miner will still be creating new blocks, which won’t be fully filled. A lot of those blocks can end up nearly empty because transactions tend to spread through the network slower than they get confirmed.

meh, I think that is an overly expensive and risky solution to a fairly modest problem. Yes, when you start a new node it doesn't have the transactions yet.  Wait a day or two.  Tada. Solved. Across brief restarts there isn't a big issue since the mempool is saved and reloaded. To improve that you don't need something as complicated as weak blocks: A node that doesn't have transactions knows it, it doesn't need transitive relay:  It could ask its peer about them.  There was the old mempool message but it has a privacy problem and didn't have the right limits to prevent abuse.   But something similar could be reintroduced that does something like "inv me your next block of transactions" which a node could run against a random peer once per block until the rate of new transactions being learned from it becomes low. If the resources for INVs is too high it could instead use some efficient block transfer mechenism -- but there is no need for POW, the requester of the data *wants* it and doesn't need anyone else to prove its good.

It really highlights the difference between what you see on platforms like mempool.space and what's happening in real time on your own node. The slower propagation of transactions and the creation of nearly empty blocks by new miners are often overlooked.

I simply agree with @gmaxwell, especially majority of time/effort spend to maintain Bitcoin Core[1] rather than adding new features. Using your example, active miners only experience such problem up to 2 days. And for people who don't mine or run node 24/7, they probably better use centralized mining pool. [1] https://www.coindesk.com/consensus-magazine/2023/02/23/can-bitcoin-afford-to-maintain-its-core-network-coindesk/

Not really, because your node can have a different mempool policy, than someone else's node. And then, you will never accept some transactions, even if accepting them is more profitable, than rejecting them. If you have the whole block, then you can compare it with your template, and clearly measure some things in the context. If you receive a single transaction, then it is handled individually, and then you have A->B transaction, and B->C transaction. Including both is profitable, but including only A->B is not, and you never receive A->B (because of your mempool policy), so you will never know about the existence of B->C, even if B->C alone meets your mempool rules. When it is reloaded, a lot of old transactions are confirmed, so it often becomes empty, and you have to fetch everything from scratch. More than that: your "reloaded" version usually contains some transactions, which were unconfirmed for a long time, and then, for example mempool.space marks them as "added", while in reality, they were just regularly broadcasted, and then dropped by their node. Which means, that if someone is not running their node 24/7, then it is possible to trace that particular node, and potentially identify blocks, mined by the same miner, even if mining addresses are not reused. 1. If you have Proof of Work, then you know, that miners are really working on that. 2. Without Proof of Work, it is possible to spam some particular node, by exploiting some particular rule of a custom mempool policy. 3. Weak blocks will be always invalid, so you can test things there. It is a solution to having a test network, where coins are really worthless, and not traded, like testnet4 coins currently are. Also, weak blocks are close to the mainnet, so you don't start with 50 coins, and almost no transactions, but you actually work on mainnet data, to scale it better. Mainnet never had "50 coins plus 1 sat/vb fees" policy. It is artificial to see conditions like that, which is why testnet4 is going to become just another altcoin, instead of being the actual test network, which follows the mainnet. For that reason, your node should relay weak blocks, made by other nodes. In this way, you don't know exactly, which node shared which block, who is a miner, and so on. Nodes can be somehow identified by their mempool policy, if you feed them with a transaction, and later ask them about it. However, those tricks don't work in case of blocks, because only transactions can be non-standard. Block data can be only valid or invalid.

Not in testnet3 anymore. If you have a stream of incoming transactions, which is bigger than (or equal to) the stream of confirming transactions, then you are "never gonna catch them all". Whether it is a good thing, or a bad thing, is disputable, but it has some unexpected consequences: 1. You can have transactions with a lot of fees, which will stay unconfirmed, because the stream of new transactions will take their place, even if the fees for new transactions are lower (because even if some block explorer will keep some transaction, then it will "expire" in other nodes, after two weeks of not being broadcasted). 2. If miners will keep mining, then their blocks will be more empty, than they would normally be (because of bad transaction propagation). Many users would expect, that if there are so many transactions, then blocks should be always full, right? Not in practice, because if a miner is in the state of "wait a day or two", then that miner can wait or mine. Waiting will allow allocating more resources to fetching transactions, while mining will result in a lot of power, spent to confirm a lot of quite empty blocks. And of course, if others are not waiting, then after you "wait a day or two", then transactions you fetched, will be already confirmed by other miners, so you will keep waiting indefinitely (until the flow of new transactions will decrease). 3. In practice, testnet3 shows us, that even if the hard-coded max block size is not reached, then still, there is de-facto artificial max block size, which is expressed as "transaction load per 10 minutes". If blocks are mined faster than every 10 minutes, then you simply have more empty blocks, as a consequence of that, whether you like it or not. If you want to increase the number of confirmed transactions per 10 minutes, then standard transaction fetching won't help you (again, it is disputable, if it is a good protection from abusing max block size, or if it should be changed, to enforce "4 MB max per block", instead of enforcing in practice "something around 4 MB max per 10 minutes" rule). As testnet4 became dominated by mining pools, focused on mining CPU-mineable blocks (which was expected, by the way), I switched back into testnet3 for a while. I wonder, if this network will ever die, because only there, it is possible to test fee-based block rewards in practice, other networks simply don't have enough halvings, or enough transaction load for that kind of testing. Also, interestingly, my initial conclusion is that it is better to wait for some time, than to mine continuously, because then, spending CPU power to mine a single block with 3 tBTC reward from fees, is far better, than mining 100 blocks 24/7, and getting 0.01 tBTC for each. But obviously, the right time between waiting and mining, is somewhere in between, and I am trying to find that equilibrium (you cannot wait for too long, because other miners will pick those transactions, and you cannot wait for too short, because you will spend a lot of CPU power, for mining almost empty blocks).