I checked out the paper "Bandwidth-Efficient Transaction Relay in Bitcoin" (the Erlay Project) and honestly, I think it’s got some serious flaws. I might have a better idea to tackle this.
Let’s rethink how we do full mempool reconciliation. We could use an AI model for this job trained on a CPU. The training times on a regular CPU are pretty telling:
0.147s for 100 transactions with 100 inner Merkle tree nodes
3.984s for 1,000 transactions with 1,000 inner Merkle tree nodes
A whole lot longer for 5,000 transactions with 5,000 inner Merkle tree nodes
Now, if we apply bucketing (basically sharding based on the initial hash byte that breaks the task into 256 "equal" subtasks) on that same CPU, check this out:
0.004s * 256 = 1.024s for 1,000 transactions, 1,000 inner Merkle tree nodes, training 256 models on 8 data points each [48 * 256 bytes model]
0.029s * 256 = 7.424s for 5,000 transactions, 5,000 inner Merkle tree nodes, training 256 models on 40 data points each [157 * 256 bytes model]
0.057s * 256 = 14.592s for 10,000 transactions, 10,000 inner Merkle tree nodes, training 256 models on 80 data points each [258 * 256 bytes model]
Surprisingly, this method is faster. Plus, it’ll get better as CPU technology improves. I’m sure mining farms wouldn’t mind this.
When Party A sends this 66KB (more like 44KB in reality) model to Party B, Party B can quickly reply with just the transactions that Party A doesn’t have. And the coolest part? This AI model is way smaller than just a simple list of hashes. It measures about 3x-4x smaller.
Our setup (which is actually 256 models) will help predict certain boolean values based on the transactions and tree structures.
Erlay seems to have some issues here’s a better proposal for a bitcoin protocol without invites
3 replies 284 views
And my guess is that it would not be Legacy compatible so would be just another fork of Bitcoin - aka, an altcoin and not Bitcoin...
In what respect? The core reconciliation mechanism in erlay is arguably close to information theoretically optimal: If peer A wants to tell peer B about the IDs a knows and B doesn't know, it need only send as much data as the number of missing IDs, no matter how many they have in common even though A has no idea what txn B are missing.
Reconciling relay sets is almost functionally equivalent to reconciling mempools, however mempools can have persistent differences due to policy differences between nodes or due to ties in preferences between conflicting transactions. This means that there is a risk of continually leaking bandwidth due to a persistent difference in the mempool, which I think makes reconciling relay sets more attractive.
What precisely is this machine learning model learning to predict? Absent such a description your post is hard to distinguish from the significant numbers of non-technical conmen in the cryptocurrency space that just spew jargon.
The author of transactions can control the bytes of their hash, so any scheme that hopes to distribute work/load with respect to transactions are vulnerable to attack (e.g. attacker mines flood of transaction that all begin with a common byte).
That is a substantial amount of data, several times what the Bitcoin network uses today to relay a block in the presence of common transaction, which itself is substantially larger than we know is possible (existing mechanism is used for implementation simplicity and to reduce the latency impact of decode time).
Checking a friend's node, just to given an example, I see that the last 7 blocks took the following number of bytes to relay to this node: 38227, 18617, 30581, 31484, 41335, 42155, 24038.
Transaction order is a property of both the dependency graph and the miners selection algorithm, as a result attempting to reconcile on interior nodes will mostly just waste bandwidth because even with the transactions are the same the alignment in the tree is often different. Efficient reconciliation protocols effectively don't use any bandwidth for common data, so there shouldn't be an advantage on reconciling on interior nodes.
This and the following text has no coverage for if it's not a block but a new transaction. Erlay is *not* a block relay protocol. It is a transaction relay protocol, and given my above efficiency comment I don't think what you're imagining is particularly interesting as a block relay protocol compared to what is already deployed.
Exactly. Mining a single byte in transaction ID is very easy. There are transactions with four mined bytes, for example: https://mempool.space/tx/000000000fdf0c619cd8e0d512c7e2c0da5a5808e60f12f1e0d01522d2986a51
?Reply
Sign in to reply to this topic
Related topics
- Ways to earn some sats by contributing to bitcoin core development 5
- Introducing cBTC: A Bitcoin-Backed Monetary Protocol Concept Seeking Feedback 19
- issues with bitcoin core database read error 2
- Issues connecting Bitcoin Full Node 3
- Bitcoin Core displaying coins (UTXOs) in a new tab 13
- Are you in favor of BIP-110? Let's get a Bitcoin poll going. 0