webhintio/online-service

Permanently delete url generated from webhint online scanner

bhavik09071990 opened this issue · 4 comments

bhavik09071990 commented

We have used online webhint scanner to generate report for our website.

I would like to know if it is possible to permanently delete footprints of our website, or the url which was given to us by online webhint scanner. I would like to permanently delete that URL or delete any data specific to our website. Let me know if it is possible.

molant commented

Because anyone can submit any url we have to be careful on how we do the deleting. I can think of 2 things:

  • Opt-out of scanning. We could have something a la robots.txt that tells webhint not to analyze the website with the online scanner. This will be useful for websites that do not want anyone analyzing their site. This is good to prevent future scans.
  • On top of that, we could add a form that looks for that file/configuration and if present deletes all the previous ones.

@antross pinging you because we were discussing this earlier.

sarvaje commented

Opt-out of scanning. We could have something a la robots.txt that tells webhint not to analyze the website with the online scanner. This will be useful for websites that do not want anyone analyzing their site. This is good to prevent future scans.

I see one problem here, how do you know that I'm the "owner" of the url I want to block?, What if I go and I block www.bing.com? What about a bot that blocks all the urls? (because people like to do these things).

Delete a results is one thing, but allow people to block some urls I think is dangerous.

molant commented

Actually is the opposite. We shouldn't remove any result because we cannot verify they are the owners of the website but if the website has a robots.txt similar to the following then we are sure that we shouldn't scan that website:

https://example.com/robots.txt


User-agent: webhint.io
Disallow: /

We can be pretty sure that no other than an admin of that website has added that robots.txt (or whatever file we want to add) and we should respect that.

sarvaje commented

ahhh ok, I though you were talking to do that in our side. Then it is ok.