Can I delete my content from the network?
Closed this issue · 35 comments
If you add data to the network, and another node chooses to rehost it, there is no way to cause them to delete it from their blockstore.
It is possible for nodes to follow blocklists and to agree to follow common ones. But attempting to take back content is not fundamentally doable (there is no way to verify information deletion).
Could a fore encryption bit be set the next time that client comes online? This would achieve the same thing. (Ideally, I don't want this to be doable, but better think about this now before some one thinks up this "good" idea.)
doesn't issuing a "DELETE" delete content, or at least request the network to delete a file or directory to the extent that it can?
from http://ipfs.io/docs/commands/ :
-writable bool - Enable writing objects (with POST, PUT and DELETE)
@alexpmorris no. that's just REST translation for one local gateway.
The permanency of content will surely pose some legal and ethical issues. It seem like copyright holders wouldn't be able to take action against infringement and I couldn't do anything about someone publishing my nudes. I know this is already true of torrents, but if IPFS is used for web apps I imagine it being a bigger deal. What are your thoughts on this @jbenet?
On the IPFS homepage, it is described as IPFS is The Permanent Web so deletion sounds like an anti-goal to me :)
To be clear, though: it is possible for data to be deleted from the network. For example, if you add a file and then immediately unpin it and then garbage collect it before anyone has a chance to get it from you, it will be effectively deleted.
What about drive storage space if you can't delete? Who's holding (or more importantly, paying for) all this expired data and what's the incentive to do so? My idea is to use IPFS to backup my data (after I encrypt it), and offer local storage as payment (or maybe partial payment... not totally sure how that works). But I will eventually delete some of that. It would serve no benefit to anyone to waste storage after I no longer need it.
What am I missing?
Even though IPFS is for permanent web, It is required to be able to remove specific illegal contents. I think it is possible to remove them by mapping the content with peerID, and other peers that host the file need to remove the file on receiving deletion request through gossip protocol. Of course, I know that it is impossible to remove whole replicated contents, but it will be gradually faded out someday.
I guess a possible solution to filter out illegal content is to ship IPFS with hardcoded blocklist addresses, which will be updated lists of content that will never be downloaded or retained by the nodes. These blacklists will of course only contain pointers to illegal stuff.
Then there could be other blocklists depending on which laws apply where the node physically is.
Criminals could still modify the source code to access illegal data, but it's not a big deal since anyone else won't be sharing it.
@whyun7892 There is a plan of opt-in block lists called badbits. They will contain hashes of illegal contented as for some authority. For more see: ipfs/notes#21 (comment) #36
But there will not be option to just delete content from IPFS, same way you can't delete content from WWW, if someone care enough to host it, it will be there.
@Kubuxu Great. The opt-in blocklist method will take effect if some agency(such as DMCA) keeps requesting to remove particular contents on behalf of users or authorities. But, one concerns... Many blacklists from various authorities need to be managed (and by whom?), and the number of blacklist will be growing accumulatively because they will not un-list it..
If a user uploads private file by mistake, It will be more reasonable to give a chance to try to remove the content, even though it may not be completely wiped out in the overlay network, just like WWW.
If IPFS supports CRUD, it will be much more useful in various business area.
Fade-in, fade-out with user's decision :D
You can remove content you added to your node, you just can't be sure that someone isn't already rehosting it and remove it from there.
Size might be problematic but it is up to block list creator to manage it. Bloom filters can make it quite efficient to check if item is in blocklist. Many issues were discussed in issues I've linked.
@Kubuxu Thanks. Understood. :D I think you mean that it is possible to remove by use of 'pin rm' and 'gc'. right? If so, why not put 'remove' to basic commands list.
IIRC it is absent by design: remove is ambiguous (remove from what? node? entire network?), having two explicit commands for removing a pin and triggering GC leaves no room for misunderstanding of what is happening.
I understand that IPFS is nick-named "the permanent web", but surely that shouldn't mean that it is a good idea that we can never being able to delete something properly? I quite like the ideas of privacy that Diaspora ( https://diasporafoundation.org/ ) were working on, where users controlled their own data, and could always rescind it causing the distributed network to delete that data. I think that if IPFS is to become the new web, then privacy has to be one of the central features that can be chosen when needed. Clearly there will be items that need to be kept alive forever for everyone's benefit, but equally, there should be an optional concept of privacy built in so that if someone wants a picture of them "forgotten" this should be possible too. It surely about thinking of this as the underlying platform for everything, then permanence isn't always the over-riding need. It is about balancing the needs of privacy and permanence to get the right combination. Obviously a self-managed social network like diaspora could be re-built on-top of IPFS with its own controls for social data.... but at the file level, I think a similar feature is potentially desirable inside IPFS but for any files.
@emardee deleting data from any network is a gentlemen agreement, all listen and all of them cannot lie, you might ask everybody to remove the file but some of them will say "Removed." but they keep it. IPFS isn't worst in this matter than normal Internet, if you make something public to the network, you have to know that it can become permanent. Nobody can prevent anonymous internet people for publishing it all over again.
It isn't just because: the aim is so everything is permanent, there just isn't viable solution for this problem.
Diaspora is something else, it is closed (registered) community that can enforce the gentlemen agreement to remove the data.
@emardee
Remember that we can't truly "remove" things from the internet.
If you put data online there is always a possibility it will live forever.
Period.
You can issue a "revocation" requests, but in the end you have no control over them being respected by other parties. Things can be passed over alternative protocols or just saved to a cold storage to be released years later.
If you don't want people to look at your bytes -- use encryption BEFORE you put them online.
@csharpner said
What about drive storage space if you can't delete? Who's holding (or more importantly, paying for) all this expired data and what's the incentive to do so? My idea is to use IPFS to backup my data (after I encrypt it), and offer local storage as payment (or maybe partial payment... not totally sure how that works). But I will eventually delete some of that. It would serve no benefit to anyone to waste storage after I no longer need it.
What am I missing?
- You can't force other nodes to delete stuff, but you can choose what to keep on the nodes you control of course
- At the moment there is no incentive to hold data except having the guarantee of being able to access it
- in the future you'll be able to pay other people to host your data using filecoins
@csharpner
I guess the other thing you might be missing is that your encrypted files might well outlast the integrity of the encryption technique used to protect them. If you release your data to the public, and copies are kept way beyond your expectations, eventually someone might crack the encryption method rendering your data accessible to all. You'd want to be happy that by the time the encryption was likely to be broken, there was no privacy risk left in the data being exposed. For that reason, you might want to give this plan more thought. Probably better to use a trusted friend or two, and you all keep a private copy of each other's backups (but not publicly), so that you trust everyone who has access to the data to delete when required, and to not open old files if encryption gets broken in the future.
If you don't want people to look at your bytes -- use encryption BEFORE you put them online.
It is also possible to restrict who can access the files you add in IPFS. When you add a file, it's only available on your node at first. You could configure your node so that the file is only accessible to few other selected friends on the network. If they are trustful, these friends should not propagate the file further.
This could be interesting especially in the case where a crypto algorithm gets broken years later. You may not want your private conversation to be public then.
I think this is also discussed elsewhere, see: ipfs/kubo#1386
You can neither unsay nor unhear something in the real world. We need to regard the hash link as its contents. If someone has the link they should be considered to have the file. To delete the file you need to remove all references to it, like garbage collection in programming languages. So we need to convince the legal system that having or publishing, a hash link, to be the crime, while actually having the illegal file that the hash links to on a server to not necessarily be illegal.
@fazo96 There is no global understanding of what is illegal and what is not. Just one example: Publication of Nazi insignia outside historical context or research/education is illegal in Germany but perfectly legal in the USA as far as I know.
@ingokeck True, but it just means that we need multiple blocklists. One of the first will probably be a blocklist for DMCA takedowns: someone will maintain a list and publish it via IPNS, and all public gateways will opt into it. Then if another list needs to be made for any other reason, it's gonna be made and used.
Is it possible to restrict content prior to accessing it on IPFS? Lets say I'm hosting a file on my node. I know exactly who should access my node because I have their addresses whitelisted on an Ethereum Smart contract.
My contract wrote Adress A = 0xA.., Address B = 0xB.., and Address C= 0xc... My IPFS node hosts another smart contract and a file. The smart contract prompts MetaMask upon arrival at my content and checks my Address. My address is 0xB, so I am whitelisted on the contract. If i was 0xD, could my contract stop IPFS from sending the content to 0XD?
Hi, read all this and still don't understand if old files can be deleted or not? I have a network of media players which I'd like to share download responsibilities. Is old content able to get deleted or not? If the disks are going to fill up in a few days ipfs is not a solution for me. ;-)
Files can be deleted from your own local instance(s), but you cannot force remote nodes that already have a cached copy to delete their copies.
I think the idea of having the ability to "remove" is part of the problem with the web. As long as there remains the possibility of "Taking Down" stuff, then it will create a situation where those that want things taken down will over-extend their power. On the other hand, I think where people should have more ability to change things is at the input point--i.e. if your friend has taken a picture of you or something it should notify you somehow that the picture is taken, and then you can decide whether or not your friend gets to keep the shot.
I think Etherum has the right idea as far as contracts go for consumables like movies.
My understanding:
{Your [encrypted] }files will be deleted when the action of accessing them ceases (metric for relevance & demand necessity / 'data stockpile' / 'warehousing' criteria is met); if, nobody else accesses them; and, they are cleared out in retired 'junk' cache by incoming data demands.
Otherwise - What you post ... is Posted.
Web Apps will Update with new URI's = old data not accessed anymore = deleted if nobody cares for it.
Otherwise; look for IPFS to be a new host to torrentz of leakz - "Hillary Under the Cover(s)" & "Trump's Rumps" e-mails & failed steak campaigns which can never be unseen; much like the IPFS, a torrent inspired web.
Why This Thread Should End Here
"The Permanent Web"
'deeMCA is not in the house; everyone grab a mic & drop it.' - BlackLists
Ok, the thread shouldn't 'end here'
The notion of a private network which has agreements to host files is interesting.
Paid 'club' membership - or to individuals - or geolocalized nodes...
.
This would require control over distribution to networks.
-> https://news.ycombinator.com/item?id=10329154
& xanadu is still cool; good to see it 'roun'.
@mixmastamyk You don't have to bother with your local storage filling up. Just don't pin the content. The local storage only acts as a limited sized cache, unused data will be discarded.
Regarding the rest: Part of this could be solved by deploying encryption by default. If you want to share with everyone, just don't encrypt. Otherwise some sort of key chain is needed that ensures only allowed parties can view the content but it would still be distributed accross the ipfs network.
BTW: Does the ipfs ever forget? I mean, after all it should be permanent. But if nobody references the files any longer, and nobody requests them, they should fade away, do they?
Regarding DCMA requests and blacklists: Is this really a usable solution? I could change one bit in a video stream, that wouldn't hurt the stream at all. But the hash would be completely different. This smells like the myth of Sisyphos... It would result in requests to detect offending files by fingerprinting, would there be a solution using fingerprinting? Maybe by distributing (and enforcing) additional metadata with the files...
This issue was moved to https://discuss.ipfs.io/t/can-i-delete-my-content-from-the-network/301
Let's say I decide to create a dapp and store its files on IPFS, but since there's no way of deleting, every time I want to push a change to the app I would have to deploy a new one, is that right?
Without data deletion/pruning, isn't storage space a problem for nodes over time?
You can delete your content from your IPFS node, you just can't force another IPFS node to delete something they've downloaded from you.
Just to clear up any possible confusion, IPFS doesn't automatically distribute/replicate content. Content is only stored by nodes that choose to download it (and then choose to not delete it). If nobody wants to store the content, it'll disappear.
IPFS is permanent distributed web.
Although data can't be permanently deleted, maybe we can remove the content & update IPNS with the new content?
It is a sub-optimal approach, which may work.
@jbenet @flyingzumwalt @Kubuxu @whyrusleeping , Please confirm the feasibility - and maybe we can add a remove feature, subjected to the condition that the file/content is in the IPNS zone.
You can make your own IPNS address point to anything you want (including an empty directory or file). However, this doesn't actually remove the data from other nodes.
You can currently remove data from your node by:
- Making sure your IPNS address doesn't point to it.
- Make sure you aren't pinning the data (use
ipfs pin rm ...) and it's not stored in MFS (useipfs files rm ...). - Run
ipfs repo gcto remove all unpinned content.