I've been thinking about how we can build useful applications on top of Bitcoin. It reminds me of Ross Anderson's concept of the Eternity Service, which Adam Back once implemented with Perl on Usenet years ago.
Usenet has some durability, but it doesn’t hold a candle to the Bitcoin blockchain, which is backed by millions of dollars in value. Just think about how tough it would be for any government agency to pull down the blockchain. Plus, every user has a complete copy of the blockchain, so anything stored in it has a level of plausible deniability, similar to how NNTP servers work with full feeds.
I get that some devs might argue Bitcoin should only be a currency and freak out at the idea of it being used for file storage. But honestly, new applications for internet protocols pop up all the time. Trying to stop people from saving small, valuable datasets on Bitcoin is about as effective as telling folks to stop sharing copyrighted stuff on Bittorrent.
So let’s break down the costs of storing files on the Bitcoin blockchain.
We should probably aim to keep blocks under 10,000 bytes. A good spot for random data would be in the address fields of outputs. If we throw down a fee of 0.01 BTC, we can keep output values below that amount and have as many as we need. Using 280 outputs as a nice round figure and setting their values to 0 BTC, we can fit around 5800 bytes of arbitrary data into a block that’s about 9712 bytes long, all for just 0.01 BTC.
I can see many uses for this, but consider this if you will (You'll need your tin foil hat)
if you can encode images into the blockchain, and everyone downloads the whole blockchain, then, by causality, if an illegal image were to be written to the blockchain (you all know what I'm talking about), and we download the data, would this be grounds for govt action against any user of bitcoin?
An interesting idea, and encoding the bitcoin logo into the chain is a funny easter egg But please reconsider using the block chain as a generic storage device for data. It is already growing quite fast as a transaction log alone. If people start using it as a file storage service, rates will have to increase to account for storage costs and the people using it for simple payments suffer.
Bitcoin is an incredible idea in itself, there has never been a distributed virtual currency this effective. In my opinion we should keep that idea pure and not put in auxiliary unrelated things.
I mean what will happen if 1000 people get your idea and start encoding images, or maybe even video into it? It will take ages for new users to download the block chain because it is full of cruft.
Maybe you can fork your own block chain to do this and link it to the main chain somehow?
I have to disappoint you, but the bitcoin blockchain is not even intended to be a permanent storage device of financial transactions, let alone arbitrary data. The current implementation indeed lacks the ability to prune obsolete parts of the chain, but this won't remain the case. At this point, it's hard to guarantee that 0-txouts won't be pruned.
[mike] explained in http://bitcointalk.org/index.php?topic=7219.0 why the block chain is not meant for arbitrary data storage, and showed an alternative for linking other chains to it, with almost no overhead.
For a network that was designed as an anonymous data-storage system, see http://freenetproject.org/.
The point I'm trying to make here is that on one hand, we have all these great theoretical ramblings of the robustness of Bitcoin, and "Proof of Work," and the massive amounts of computational resources it would take to subvert the system.
On the other, we see constant references to people playing nicely, and "not intended to be," and "contrary to the wishes of the community."
Both these things can't be true.
What happens to the stored value of the block chain when something completely unacceptable gets steganographically encoded into it, and serious resources are mobilized to prevent its wider dissemination?
The government has taken down botnets before. Seized their IRC server domains, and gone around deleting the software from individual machines in the seed list of peers. What happens if the developers are ordered by the courts to distribute a trojaned client, and face long prison terms if they publicly let on that there's an investigation.
The idea of Bitcoin as this pie in the sky distributed system with no single point of failure, totally immune to the actions of governments, is silly. They used to say that about Usenet, and the government has by now managed to terrorize most significant players in the ISP business into not running news servers.
Suppose that instead of encoding a Bitcoin logo into the block chain, I had encoded the names of all the CIA Station Chiefs in problematical countries, or nuclear missile launch codes. Do you seriously think your P2P block chain wouldn't get squashed by a large boot?
So while I'm not seriously planning on offering a Bitcoin file upload service, I do think the notion that Bitcoin is invulnerable, distributed, and won't wind up like eGold and Liberty Dollar once it gets big enough to be on the federal radar, is pretty naive.
Meanwhile, people can probably upload that DVD Screener of Thor to the block chain before you can upgrade enough people to a client that doesn't allow 0-txouts.
Vanity, Vanity, All is Vanity.
I don't know when you checked last time, but we're not living in some kind of James Bond novel. It'd be funny though, CIA cowboys hunting all around the planet to burn all last remains the P2P chain. After all, they were so successful in shutting down Wikileaks.
We don't really have a use for another thread full of "what if?" scenarios. I'm pretty sure all of those have been addressed to the point of boredom before. Just do a search.
While some providers like Amazon and Mastercard have made a decision to decline a business relationship with Wikileaks, the government has not tried to cut off their connectivity, nor to expunge all copies of the problematical material, nor would it necessarily be legal for them to attempt to do so.
The government has, on the other hand, managed to completely destroy some famous international botnets.
Bitcoin looks a lot more like a botnet, than it looks like Wikileaks.
Now granted, if you convert dollars into bitcoins, and buy things with them relatively quickly, you take little risk that the axe will fall while you are holding the bitcoins. I think the risks of using Bitcoin as a long term store of value are underappreciated.
Once the government realizes that Bitcoin can be used to buy naughty things, it will probably follow the same route as eGold, which could also be used to buy naughty things.
Bear in mind that money transfer services are legally required to be pro-active with regard to suspicious activities by their customers. Bitcoin totally lacks any ability for that sort of oversight. Accounts can't be frozen, and transactions can't be tracked.
I'm sure that sooner or later, things like this are going to become an issue.
Don't botnets generally have a centralized command and control system? Isn't that system what has been taken down in the past? I don't think that Bitcoin has any fundamental components vulnerable to such an attack. Right now there is the issue of using a single port and bootstrapping from IRC, but if we found ourselves in an arms race against a government, I think we would win.
If you read what sipa posted, its not just that it is "contrary to the wishes of the community" but also, it doesnt work. There is no guarantee that transactions wont be pruned from the blockchain in the future (in fact the almost certainly will be in the fairly near future). So storing data there just isnt reasonable. If you just want to transmit a message, it would probably work, but if you want permanent storage...not so much.
Perhaps. I think it's in everyone's best interests that the Bitcoin vs. Government battle take place sooner rather than later.
Having it happen years down the road is a very bad idea, especially given how usage and bitcoin value seem to be taking off in leaps and bounds.
To us, it's Bitcoin. To the System, it's a money laundering botnet that can be used for anonymous purchases of illegal drugs and other nefarious things.
Otherwise, we risk waking up five years from now, turning on the TV, and hearing that hundreds of members of an "International Crime Ring" called Bitcoin were arrested in coordinated raids in a dozen countries.
The time has come to poke the bear with the stick, and inventory the bear's offensive capabilities.
This is incorrect.
Information storage transactions with a non-zero balance can only be pruned after they have been spent. Don't spend them? They stay forever. Any suggestion to prune unspent transactions is likely to be unpopular.
When do you imagine is this "fairly near future" in which transactions will be pruned? The network protocol has to be changed substantially to facilitate pruning. I imagine that converting the majority of the network to pruning clients will break the old clients.
If you ensure that your transaction hashes to a certain value then you can store information in the block chain (on a probablistic basis) even once the transaction has been spent and pruned. With the low height of the current merkle trees, this would be quite reliable.
ByteCoin
Tx deletion does not require protocol changes*. Any client can delete a transaction at any time, they simply run the risk of being unable to verify transactions in future if a re-org takes place. If the transactions are buried under enough blocks this risk becomes low enough to not be worth worrying about.
However, if the transactions are not financial in nature then they will never be spent (as they have no corresponding private key) so they can be deleted immediately without risk.
This technique is much discussed, but if encoding arbitrary crap into the financial block chain becomes common I'll just write a set of patches that make it easy for people to delete those transactions. If the files aren't encrypted then they will be trivial to identify and automatically remove. If they are encrypted then you have to distribute the keys somehow. Widely distributed keys run the risk of being discovered by an honest node at which point the bad transactions become identifiable and deletable. It's very likely that most nodes will choose to delete non-financial data if it's easy enough because it costs them to store that data, yet they get nothing for it - indeed their resources are being abused for something they never signed up for.
So whilst it might be possible to hide small amounts of private data in the block chain for a price, I doubt it will ever be an attractive way to store data relative to other services.
Enochian should consider that there are much better ways to help Bitcoin than broken window fallacies. Perhaps his or her energy can spent more productively. If you really want to build a high-difficulty file storage chain I've already explained how to do it.
*I originally said no protocol changes are needed but was wrong. The block message would have to change to be sent as a header+list of hashes instead of header+full transactions. It's an easy change that will probably happen anyway as it's a nice bandwidth saving
The "fairly near future" I was referring to was not holding the blockchain on the majority of nodes that is fairly high on the to-do list. Also, pruning spent txouts on miners will probably be added soonish as well. That just leaves people who "pay" to have their data stored by creating transactions which they won't/can't spend. However, as [mike] pointed out, the majority of these methods are identifiable, and any method where you go out of your way to not have them be identified would have a pretty decent overhead/cost per data.
Small encrypted high value data sets are indistinguishable from addresses, and the keys would almost certainly be distributed out of band, so there is no way to distinguish such transactions in the block chain from financial ones. Since the outpoints will never be spent, they are not candidates for pruning should reaping the merkle tree ever be implemented.
The only practical way to prevent the block chain from being used as a data store is to make it too expensive to do so. Prohibiting micropayments would be one way of doing this. Making the small output fee "per output" would be another.
I'm not sure what pointing out obvious vulnerabilities has to do with the "Broken Window Fallacy," which is the assumption that damaging infrastructure boosts the economy by the cost of repair.
Were a war of attrition to occur between persons wishing to use the block chain as a persistant store, and you writing patches for the client, I would put my money on the file sharers. Bits, after all, yearn to be free.
If the keys become known then everyone can prove to themselves that those transactions are part of a file and can thus be safely deleted.
If you can keep the keys secret and away from anyone who actually runs a node, you might be able to get away with it for a while assuming there are no other tell-tale giveaways. But storing data in the block chain means:
It has to be encryptedIt is expensiveThe people storing it have a financial incentive to destroy the data if discovered
Such a system could hardly be described as an "eternity service". It is actually rather useless. If you have a small amount of encrypted data you want to share with a small number of people you might as well just email it to them. Sending messages via the block chain means you need to tell the recipients the keys and where to look for it ... and if you have a way to do that, you may as well just give them the message that way too.
I don't need to keep the keys secret from anyone who runs a node. I just need to keep them secret from the very small number of people who understand the client code well enough to modify it without breaking it, and also disapprove of non-financial data in the block chain.
One obvious thing for people to store in the block chain is their encrypted private keys. That's only a few thousand bytes, and after doing that, they can burn their PC and their wallet, and get everything back on a completely fresh machine, and never have to worry about losing their coins.
That's certainly worth 0.01 BTC, and only one person needs to know the key.
It's a requirement that clients support the downloading of the blockchain for new vanilla clients connecting for the first time. If you delete transactions, this breaks.
You presume incorrectly that because you can identify where the information is stored, you can delete the transactions with impunity.
At the cost of some storage inefficiency I could create a scheme using error correcting codes that stores a file in the block chain which would rely indistinguishably on some innocent transactions (not party to the file storage scheme) and a majority of file storage transactions. If you wished to delete some transactions to prevent the recovery of the file you would run a grave risk of deleting some innocent person's unspent transactions. I believe the odds could be made rather bad for you.
Suppose the file is stored using equal amounts of innocent transactions, indistinguishable storage transactions and indistinguishable error correction transactions. You'd have to delete a third of the total transactions to render the file unreadable. The chance of deleting an innocent transaction is high.
Also, deleting transactions doesn't stop people storing data in the merkle tree stubs as I mentioned earlier.
You have not shown how the file storage chain can benefit from the Bitcoin chain without substantial invasive changes to Bitcoin.
If I am wrong, please clarify in the relevant thread how your scheme achieves this and sent me a private message with the link and I will change this post.
ByteCoin
Which is going away (hopefully) very soon.
No, he's quite correct here. If miners don't mine your txins, it causes no problems for anyone but you, hence if you can identify where the information is stored, you just render those coins undependable, which, as they will never be spent to avoid pruning, is not a problem.
I disagree, if you are using error correction checks, it means you need to store even more data in the chain. This gets prohibitively expensive once you start storing large amounts, assuming you want to be 100% sure you can't identify them from other transactions. However, there are other interesting methods of identification like time, similarities, and such, but those could be avoided if you work very hard.
Also, remember that [mike] works to prevent false signups on Google account services so telling him he is wrong about what you can and can't discern statistically is probably a bad idea...(especially if a medium false-positive rate really isnt much of a negative side effect)
And if you purge the merkle tree as well? There is nothing stopping you from purging branches of merkel trees after their txes have been purged (again, works short term but not so much as "Eternity Service")
To an extent, you are both right here. It has been shown how to do it properly and create a storage chain. And it does require fairly-minimal changes to bitcoin miners (only those who wish to support this new chain). But it requires no changes to the majority of the network.
You can't prune unspent 0-value outputs because they can be redeemed, even though this would be useless. You can't necessary prune unspent known-to-be-data outputs because the script can both contain data and be redeemable.
Detecting data transactions based on entropy would be error-prone, since true randomness has a chance of being apparently low-entropy.
Deleting redeemable outputs is dangerous, since you could end up in a situation where you need the output to verify a block, but none of your peers has it. No matter what you do with the block, you might end up on a separate chain.
Yeah, I guess it depends what you mean by invasive changes to BitCoin. It does mean changes to the software, but only a couple of new RPCs really and a few tweaks to the getwork protocol. The changes sipa is making to handle wallet sharing, per-txout spent tracking etc are far trickier and more invasive IMHO. Which changes were you thinking of?
At some point somebody will add the necessary RPCs and put together a little framework for the alt-chain side, then I think a lot of these sorts of discussions will peter out because it'll be so easy to do things the right way. Heck, maybe I'll do it, if I get some time this summer and Vince doesn't beat me to it.
With respect to ECCs and such, I think it's worth taking a step back here. To fend off abuse you don't have to make it impossible. That's hard because there's always another move that can be made. To win all you have to do is make abusing your systems less appealing than using the alternatives (eg, freenet).
For storing files in BitCoin specifically, it's already pretty unattractive and that's with nothing more than the IsStandard checks. Making it harder is easy - we already established that you'd have to encrypt your data, which means there needs to be a side channel telling the recipients which transactions to read and what the keys are. If you have such a side channel, you probably don't need the block chain at all.
What's more to receive the file, now everyone who wants it has to download and process the entire block chain, a process that already takes forever just for financial data.
So it's expensive, slow, complicated .... I can't forsee a time when BitCoin makes more sense than Freenet or BitTorrent, etc.
There might well be schemes where this sort of thing makes sense, yes. To be clear, I don't object to storing things that are essentially financial in nature in the block chain - that's what it was designed for and it's good at it. In future we'll see transactions that represent multi-party contracts and such.
This specific idea doesn't make sense because if you encrypt one set of private keys, you end up with just a different set that also have to be stored. If you then burn your computer you'd lose the second set so you're no better off than you were before.
I think what you want to achieve can be done with password protected transactions. It'd need to be carefully designed to make brute forcing very slow, but if you did the work to support such transactions, it'd certainly be interesting and worth considering for inclusion.