Bring back preset rules
gorhill opened this issue · 37 comments
In a different, more flexible form. For example, we want regions to be able to create their own rules, and for users to import community-contributed rules, with no dependency on the Github repository.
Not sure if I will wait for this before shipping though, it's a big item.
I believe this could be accomplished later to appeal to a wider audience when you're close to a 1.x release, though I must state that this is just my opinion.
By the way, excellent work @gorhill! Loving the changes thus far!
New idea: Some sites integrate with Google Drive or Github. Is it possible to "inherit" rules from other domains? So if a web-app integrates with Drive, you can have it inherit Google Drive privileges and communicate unhindered?
This is just an idea, I haven't fully thought it out yet (how it interacts with global or 1st-party rules, etc.)
I believe too that uMatrix could be better. Like said a lot of website integrate YouTube (etc). So, via one click on the button, uMatrix should be eable to create (directly) the right rules for the media (etc). Only a few rules are enough.
It would be useful because one user simply need somtimes to create temporary rules. By default, it sould be temporary. After, the user confirm (save).
I block everything. Then, no I don't want to create a global rule like :
* ytimg.com image allow
Also, it will simplify the process when a user start from scratch. I think about regulary, since a very long time (>6 months). It could be nice as a new (or back ? Recipes ?) feature and perhaps gorhill the day to be brave is not far...
One example below :
malwaretips.com s.ytimg.com css allow
malwaretips.com ytimg.com image allow
malwaretips.com s.ytimg.com script allow
malwaretips.com www.youtube.com script allow
malwaretips.com www.youtube.com xhr allow
malwaretips.com www.youtube.com frame allow
malwaretips.com googlevideo.com xhr allow
malwaretips.com redirector.googlevideo.com * block
-----------------------------------------
## Comment
## It's works fine
* * * block
* redirector.googlevideo.com * block
-----------------------------------------
Perhaps there's also something about (the damn) Recaptcha :
https://www.google.com/recaptcha/
## By the past : https://www.gstatic.com/recaptcha/
## To complete one day...
## Recaptcha doesn't figure on the url, it's loaded by two websites (google.com and gstatic.com).
* www.google.com frame allow
* www.google.com image allow
* www.google.com script allow
* www.google.com xhr allow
* www.gstatic.com image allow
* www.gstatic.com script allow
I've been using an extension called Stylish lately to adjust C.S.S. It has a "style template" sharing interface that is very convenient since it is only two clicks and provides a nice preview. Perhaps this could provide a good reference? I have described what I imagine below. This would be designed for quick adjustments and sharing, not for bulk scope management. Bulk scope management is basically already provided by the "Options>My rules" page, although it might be useful to separate imported rules to be committed, at least temporarily.
A single button, perhaps left of Refresh in the drop-down, could open a dialogue or simply change the drop-down panel to review/edit the current scope before uploading. This would ensure users know what is being submitted, doubling as confirmation, and allow them to add comments and name/describe the rule set.
For those installing, they could select from a list of names or use arrow buttons to preview the new rule set before applying and refreshing (one button). If rules are previewed in the normal matrix panel, comments could be displayed above, or floating below or to the side depending on the display size. This would allow users to read the notes, or any other non-matrix rule information without opening the settings menu; perhaps they should only display when previewing.
Theoretically, I don't think it would be necessary for templates to be explicitly named or described since the scope and rules would clearly identify them and comments could be used to indicate the rules' purpose. The only problem I imagine would be those rules for domains that have not yet been referenced by the active page, which would have to be added to the matrix for preview and might be a bit confusing. Realistically, I don't know how many users would properly comment their scopes so it might be worth providing a blank name field "## Name: " in the header to encourage good practice, or just requiring a name before uploading to be safe.
For translators:
- Are you sure you don’t need to invalidate the 3 additional (changed) ubiquitousRulesPageName, hostsFilesAutoUpdatePrompt & hostsFilesExternalListsHint strings for locales on Crowdin to to prevent them from remaining unnoticed?
- While at it, can you please remember not to include trailing periods for option text strings such as "Auto-update assets" and fix it?
Are you sure you don’t need to invalidate the 3 additional
It's what I meant to do yesterday when I updated the file, but apparently I did it wrong. Do you know how this can be done without having to go through all translations?
I'm seeing a new icon in the popup UI in v1.3.3.1 - https://i.gyazo.com/a05b63773ec7ffdf14aca126ea4b2627.png
The one besides the eraser, but it doesn't have a tooltip defined. What is it ?
Nevermind, inspecting it reveals it's a button for the toggling recipe rules ? Seems you forgot to add the tooltip for that.
Do you know how this can be done without having to go through all translations?
Sorry I don’t know, not even with (temporary?) managing permissions. ;) Yet there must be some way to do this quickly.
Side note: in general, the period issue is also valid for tooltips.
Seems you forgot to add the tooltip for that.
There is no tooltip for buttons which serve dropdown menus -- the tooltip would interfere with the content of the dropdown menu visually.
@gorhill, can you clarify recipe syntax? The way I see it:
name
source destination
source destination [type [action]]
Recipe is shown if exists a request matching 2nd line.
If a rule with the same source destination type and different action already exists, it is not overwritten.
action is always omitted in uAssets/recipes/recipes_en.txt, is it optional or always allow?
Are global recipes (source destination = * *) allowed?
First line is name of ruleset recipe.
Second line is condition for recipe to be a hit to current matrix:
- First must match source hostname (the current site's hostname).
- Second must match one of the destination hostname (one of the 3rd-party in the matrix).
*means match any.
The rest of the lines are rules to be imported, only on a per-need basis (if internally the associated cell is blocked). allow is implicit (I didn't see the point yet to block more stuff on top of default-deny).
At this point this all need to be field tested for me to adjust as necessary.
Are global recipes allowed?
I don't think I forbid it but I don't see the point, only users should be able to allow non-specific rules.
I am currently trying to come up with a UI to easily enter inline recipes, to be able to quickly test recipes, to encourage contributions.
remove spurious trailing periods (#30 (comment))
Thanks for fixing those. Sorry for being picky while at it, but spurious periods probably also apply to matrixMtxButtonTip, matrixPersistButtonTip, matrixRevertButtonTip, (not matrixReloadButton as this has an additional line), privacyDeleteBlockedCookiesPrompt and statsPageLogSizePrompt2 (if still used).
Additionally, "Backup to file..." should probably be "Back up to file...", as with uBlock (verb intended).
@gorhill are you going to host only the English recipe, or are recipes for other languages also allowed? I've started this https://github.com/anewuser/uAssets/blob/patch-3/recipes/recipes_pt.txt
@anewuser @warnellw @ArchangeGabriel @pwd-github
You have been contributing with ruleset recipes, and as I am trying to come up with clear guidelines about how recipes should be crafted, we should centralized here all the discussions about this. I have been thinking more about all this and with the feedback I changed my mind about some of what said before, including changes that I required. For now I roughly assembled "Contributing ruleset recipes", feel free to discuss/comment/modify.
This is almost perfect for me, the only thing I’m not sure about is this part:
_ somecdn.com *
_ somecdn.com script
_ somecdn.com frame
First, the role of * in this context is not clearly defined in my opinion, and second, I don’t understand if and then why if only scripts are need there should be an * rule. But maybe that’s because I don’t understand the * rule in the first place, which brings us back to my first point.
I don’t understand if and then why if only scripts are need there should be an
*rule
That's it, that is one of the point for which I changed my mind, hence the passage:
For any given 3rd-party, if only one type of request is needed, then the rule should allow only that one specific type.
Right, missed that this was stated above. So it clearly appears that I don’t understand the * rule, since I don’t see why allowing xhr and script at the same time would require an * rule technically. Doesn’t the * rule just imply that image and css are allowed?
I don’t see why allowing
xhrandscriptat the same time would require an*rule technically
To minimize ruleset. With out of the box ruleset, a rule with type * will be picked. Then the recipe manager will process the script rule, and when it sees that scripts are already allowed (because previous * rule), it will skip the rule. And so on for all following rules.
Consider:
_ somecdn.com *
_ somecdn.com script
_ somecdn.com xhr
With out of the box ruleset, the recipe manager will just end up applying _ somecdn.com * allow, which will cause script and xhr to both become allowed. Result is one single rule for both.
In the case where scripts are blocked everywhere by default, the recipe manager will end up applying _ somecdn.com * allow and _ somecdn.com script allow.
In the case where someone blocks all scripts and xhrs everywhere by default, then there will be three rules applied, but I expect that sort of cases to be an edge one, and whoever confirgure uMatrix like this is probably less likely to need the help of recipes to unbreak sites.
OK, make sense to me now. I think this means I should change my rules from:
* 1st-party cookie block
* 1st-party script block
to:
* * cookie block
* * script block
if I intend to use preset rules (my understanding being that using preset rules would help reducing the size of my custom rules).
Doesn’t the * rule just imply that image and css are allowed?
I've considered this the absolute minimum when creating recipes. That is, I assume someone with a very strict ruleset would still globally allow CSS/images (which the * would cover at a bare minimum, and no other resources).
That being said, my personal ruleset disallows all 3rd party resources, including CSS/images. I've noticed trackers (e.g. DoubleClick) use images for pixel tracking, so I now consider all 3rd party resources hostile.
Need a tool to one-click snapshot the rules in current popup panel into a temporary recipe in a sort of scratchpad which can then be edited/tested on the spot. The current way of creating/testing recipes is tedious.
This looks good for me. I'm interested in the global rulesets though. Especially for Decentraleyes.
A small enhancement proposal.
- The puzzle-piece menu currently offers no way to disable a recipe. One must open the Dashboard, find all the entries involved and delete them. It would be nice if we could toggle a recipe on and off straight from the menu.
- The padlock buttons next to recipes names are redundant, since the changes are immediately reflected in the global padlock button and it could be used instead.
offers no way to disable a recipe
Recipes are not an on/off thing, it's a set of rules which are imported into your own ruleset. You may have already some allow rules which are meant to be toggled on by the recipe, so of course it would be bad if uMatrix was removing rules you already had set on your side before importing the recipes.
padlock buttons next to recipes names are redundant
They are not. Some recipes may import rules which are not visible in the current scope, and without that extra padlock it would be impossible to easily save these out of view rules.
The purpose of recipes is that of convenience, to easily import rulesets -- they must not be seen as more than this. Users are still completely responsible for whatever rules they agree to import as part of their own ruleset.
Just a heads up: The jigsaw piece hasn't got a tooltip yet.
Having had a chance to play around properly. I noticed that we can import the rules into the temporary ruleset and then we have to click the padlock to save the rules, that's awesome. It allows us to the tweak the rules to our own needs and that's literally perfect. However once the recipes are imported, there's no way to undo as the eraser doesn't become highlighted. Not so awesome.
And just to add to what I said before, about global rulesets. My reasoning is best demonstrated with Decentraleyes
Decentraleyes
*
* cdn.madeupsite.com script allow
* cdn.madeupsite2.com script allow
Now I note that in @warnellw's pull request, @gorhill says that he doesn't believe users should necessarily configure uMatrix for Decentraleyes or via versa, but I'd argue that we're essentially unbreaking the internet for uMatrix users which is essentially Decentraleyes does given the sheer amount of linked everything in webpages these days. I think it's important that users that care, are in a position where they can one-click allow a bunch of CDNs without compromising their privacy. Which brings me to my segue, that's all on the allow side, I'd also like to see something on the block side, as I know I block a bunch of sites wherever possible including google's sites. So if there was a tracking blocklist for the main offenders that I could import via the jigsaw, I'd love that too.
PWD privacy list
*
* google-analytics.com * block
* googletagmanager.com * block
* histats.com * block
Now the argument would be that those are already covered by the existing hostfiles, but the issue is that you have no control over them. It's like an all of nothing on a massively large scale.
Anyway, even I've not thought of how to implement it properly, so it's just food for thought at this point in time.
Where do I put my recipes written according to https://github.com/gorhill/uMatrix/wiki/Contributing-ruleset-recipes so that they appear under the puzzle button?
@ArchangeGabriel Actually, there is an "Import" control in the Preferences, which accepts one URL per line. It adds the specified file when I press the Apply button, but the "Import" checkbox becomes unchecked, the field disappears and the list is lost (the file's GUI entry appears when the preferences UI is reopened), so while using Firefox in an unfamiliar language, I thought it was the Undo button.
@aleksejrs Firefox does not allow extensions to read from file:/// URLs. I suggest you create a Gist and import its URL, that should work. Another option is to enable the advanced setting contributorMode, this will add a textarea widget in the Assets pane which purpose is to add inline recipes.
It seems like the recipe is not updating with the update now button?
I have updated the list and manually checked the current master version and my version (youtube specifically)
Note that both versions have the version 1.0
My version:
Version at https://github.com/uBlockOrigin/uAssets/blob/master/recipes/recipes_en.txt :
Is this possibly because of the version staying the same?
Or is updating of those recipes not implemented correctly yet?
My version:
How did you look at the content of your version?
I went to uMatrix Settings > Assets > Clicked on 'Ruleset recipes for English websites'.
Which is essentially a link to moz-extension://69111b79-a6da-48be-84ef-a1ddab966d86/asset-viewer.html?url=recipes_en-0
On the top it says
! uMatrix: Ruleset recipes 1.0
! Title: Rulesets for English websites
! Maintainer: uMatrix
While the latest version also says 1.0:
https://raw.githubusercontent.com/uBlockOrigin/uAssets/master/recipes/recipes_en.txt
I don't change the version -- it's for the parser to use, it will stay the same unless the format of the recipes changes.
Shouldn't "update now" pull the newest changes though?