User scripts in Manifest V3
dotproto opened this issue Β· 68 comments
PLACEHOLDER: We have a couple of issues related to user scripts support in Manifest V3, but nothing that currently feels like a good tracking issue. This issue intends to fill that gap by consolidating discussion of Manifest V3 API considerations in one place.
@dotproto (on behalf of Google / Chrome) announced the intent to support user script managers in the WECG meeting at TPAC (#232). The meeting notes have been merged in #277 and this topic can be found under "User script manager support in MV3" at https://github.com/w3c/webextensions/blob/main/_minutes/2022-09-15-wecg-tpac.md
I (on behalf of Mozilla / Firefox) previously stated that we'd like to support user script managers, ideally in a more secure way, at https://blog.mozilla.org/addons/2022/05/18/manifest-v3-in-firefox-recap-next-steps/#comment-227473.
EDIT: The API design is ongoing. Significant updates are listed here:
-
#279 (comment) by @Rob--W (2022-09-23)
Summary of Google's "code" proposal + Mozilla's proposal for a secure "userscript" "world". -
#279 (comment) by @EmiliaPaz (2022-11-03)
Google's API proposal document, "User Script Support in Scripting API". -
#279 (comment) by @Rob--W (2022-11-10)
Analysis of the requirements for User script managers + proposal to cover cross-world communication. -
#279 (comment) (2022-11-24)
Meeting notes and summary of the 2022-11-15 user script kickoff meeting. -
#279 (comment) by @EmiliaPaz (2022-12-06)
Google's updated API proposal, PR #331. -
#331 (comment) Discussion with clarification of API requirements and renewed API design (2022-12-14 until 2023-01-12)
- The full discussion is part of the thread connected to #331 (comment). The permalinks below may be broken, so expand the discussion (multiple times) via Github's UI and search for the fragments listed below.
- API requirements gathering starts at #331 (comment) by @rdcronin
- Ctrl-F: "Ask: is there (or can we produce) an overview of how USMs are implementing script injection in MV2 so that we can ensure we're on the same page?" to find comment by @rdcronin.
- Renewed API design at #331 (comment) by @rdcronin
- Ctrl-F "Okay, this seems promising : )"
-
https://github.com/w3c/webextensions/blob/main/proposals/user-scripts-api.md (2023-02-16)
API proposal merged from #331 -
#540 (2024-02-07)
userScripts.execute
proposal to execute once at runtime rather than only declaratively. -
#565 and #560 (2024-03-13)
Concrete proposal with support for Multiple user script worlds
To recap from https://github.com/w3c/webextensions/blob/main/_minutes/2022-09-15-wecg-tpac.md, the proposed API is:
- Accept a
code
property (with a string) in the individual items of the existingscripting.registerContentScripts
API (i.e. theRegisteredContentScript type). - Require a (yet to be specified) extension permission to use this feature.
- Require explicit user consent to allow extensions to use this feature (think of Chrome's "Allow access to local file URLs" option).
- This basically behaves like
tabs.executeScript
today, except in a declarative way. - The API is primarily designed to account for the use case of user script manager extensions.
My concern with this proposal is that it re-introduces the ability to execute arbitrary code in a (somewhat) privileged execution context, against all efforts to remove this from MV3. In practice, given the absence of anything better, user script managers already execute (third party) user script code in a content script, which shares the extension APIs and some privileges with extensions. This is a suboptimal design.
I propose the following API to support user script managers, which is more secure by design:
- Support a "code" parameter as documented above.
- As a restriction, the
code
can never run in the ISOLATED world (see ExecutionWorld type), only in MAIN and a new USERSCRIPT world (elaborated below). This restriction exists because the ISOLATED world is more privileged than the MAIN world: the isolated world has access to extension APIs and relaxed security policies (e.g. cross-origin requests in Safari, and having a CSP separate from the web page (at least in Chrome and Firefox, I don't know about Safari)). - Introduce a new world, "USERSCRIPT" (next to MAIN and ISOLATED), so that extensions still have the ability to run user scripts in an execution environment that's isolated from the web page. When
code
is set, it will run in this world instead of ISOLATED.- At present, I propose only one new world. This is strictly an improvement over the current state of art. Support for more than one world could be useful to extensions, but raises the complexity of implementation, so I'm leaving that for a follow-up and consider it to be out of scope in this proposal.
- The USERSCRIPT is an isolated world (in the WebKit / Chromium sense) that shares the DOM with the web page. This means that code from both worlds cannot directly interact with each other, except through DOM APIs. When an asymmetric security relationship may exist, the MAIN world is considered to be less privileged than the USERSCRIPT world.
- User script managers may want to expose somewhat privileged APIs to user scripts (e.g.
GM_xmlHttpRequest
,GM_getValue
, ...). They currently do that by defining these APIs in the content script (ISOLATED) world, supported by the extension APIs available to the ISOLATED world. By forcing user scripts to move from ISOLATED to USERSCRIPT, these extension-defined APIs would at first lose access to the privileged APIs.- This access can be restored by establishing a (synchronous) communication channel between the ISOLATED and USERSCRIPT worlds. This can achieved with existing DOM APIs, e.g. with a pre-shared secret (event name) + custom events on shared document/window. This technique may be familiar to some, as it is a way to communicate between MAIN and ISOLATED. Although used in practice, I discourage the use of
window.postMessage
for communication because that can be intercepted and/or break web pages (for previous discussion, see #78). - In the future, a dedicated API to communicate between worlds could be considered.
- This access can be restored by establishing a (synchronous) communication channel between the ISOLATED and USERSCRIPT worlds. This can achieved with existing DOM APIs, e.g. with a pre-shared secret (event name) + custom events on shared document/window. This technique may be familiar to some, as it is a way to communicate between MAIN and ISOLATED. Although used in practice, I discourage the use of
- When multiple scripts match and have the same
runAt
schedule, scripts targeting ISOLATED are executed before USERSCRIPT. This allows the extension to prepare the execution environment before the user script code is executed. - Open question: Which CSP to choose for this USERSCRIPT world? There are multiple reasonable options:
- Same as the main world
- Same as extension
- Either of the above two, and customizable in the
content_security_policy
field in manifest.json and/or scripting API.
Not security-related, but still relevant to consider in the API design:
- Currently, the RegisteredContentScript type mirrors the input as received by the
scripting.registerContentScripts
andscripting.updateContentScript
methods. The registered items of this type is exposed to the extension via thescripting.getRegisteredContentScripts
method. Thecode
parameter could potentially be a large string, so it may make sense to not include the code by default (e.g. by adding a newoptions
object togetRegisteredContentScripts
to opt in to receiving the fullcode
)
My concern with this proposal is that it re-introduces the ability to execute arbitrary code in a (somewhat) privileged execution context, against all efforts to remove this from MV3.
I agree with this concern.
chrome.scripting.globalParams was added to solve similar use-cases however it's not ever exposed to the main world scripts. I'd instead propose a way to allow these variables to be passed into the script (perhaps cloned so that modification isn't possible) some avenues to inject these would be:
- document.currentScript.params
- import.meta.params
- Inject in a script local variable prior to execution of the script. (I believe all JS engines have the ability to easily register script local properties programatically)
I'd not expose these on globalThis/window as ideally the page would have no visibility of these parameters.
@dotproto @Rob--W thanks for addressing this now.
Require explicit user consent to allow extensions to use this feature
Will this be automatically set to on
for all existing Manifest v2 installations on update to the mv3 version?
Introduce a new world, "USERSCRIPT"
This sounds promising and will make securing a lot of functions and prototypes that can be altered by the page obsolete.
The USERSCRIPT is an isolated world (in the WebKit / Chromium sense) that shares the DOM with the web page. This means that code from both worlds cannot directly interact with each other, except through DOM APIs.
This is a problem. I'd say more than the half of the userscripts directly interact with the page's JavaScript objects and functions.
Tampermonkey at Firefox (BETA for now - the stable release will follow soon) for example uses Firefox's userScripts API which creates a context separated from the page (and the extension), but still allows access to the main world (unsafeWindow).
When multiple scripts match and have the same runAt schedule, scripts targeting ISOLATED are executed before USERSCRIPT.
π
Open question: Which CSP to choose for this USERSCRIPT world?
Userscripts should not be limited by a "foreign" CSP (extension or main world) . They are made to do things that neither Tampermonkey nor the page expect. Ideally we could add a csp
property additionally to the code
property and Tampermonkey could either introduce a @csp
userscript header that is forwarded to scripting.registerContentScripts
or use existing tags to create a CSP specific to this userscript.
I second @derjanb concern with limiting interaction via DOM APIs. This limits what can be done in terms of userscripts. Being able to mess with the webpage JS is a key point.
I also liked how Firefox implemented the userScript
API. A separate context was created but you could inject additional functions into it via a callback. If the communications between USERSCRIPT and ISOLATED is also going to be limited to DOM APIs isn't that also risky? Under this, I'm assuming that extension specific APIs are not available. How can a pre-shared secret between the userscript manager and userscript be used? The secret would need to be shared to all userscript authors? Maybe I'm misunderstanding but the more I look at this the more I think "can't webpages hi-jack this?"
can't webpages hi-jack this?
Yes. Webpages can run code before userscript managers would ever be able to by using iframes or popup windows. It has been used maliciously in the past and is now partially fixed in some userscript managers, however there is no complete 100% fix. The issue of webpages being able to execute code before extensions was reported to chromium some time ago and was marked as wontfix iirc
Currently we can execute arbitrary code in MAIN world on Chrome 107 with these code:
// background.js
chrome.scripting.registerContentScripts([
{
id: '0',
matches: [
'https://www.example.com/*',
],
js: [
'main-world-script.js',
],
world: 'MAIN',
runAt: 'document_idle',
},
]);
// main-world-script.js
window.addEventListener('message', (event) => {
// message from ISOLATED world content scripts
const message = event.data;
if (message.name === 'user-script') {
const script = document.createElement('script');
script.textContent = message.text;
document.head.appendChild(script);
script.remove();
return;
}
});
Most user scripts do not need to as complex as Greasemonkey-like scirpts:
- No
GM_*
andchrome.*
privileged functions. - No automatically update from remote url.
- Execute on
document_idle
, the DOM API is accessable.
Just like users execute scripts in the DevTools "Console" or "Source -> Sinppet", but use extensions to execute scripts automatically base on the page url.
We can create a lite version user script manager with Manifest V3 now.
But does the Chrome Store policy allow this usage?
Hi everyone!
My name is Emilia Paz and I am part of the Chrome Extensions team. I'll be working on providing user script support in the Scripting API for MV3, as discussed in this thread.
I started an API proposal doc that takes into account the points discussed here and in offline conversations. The goal is to align on an API design that works better for everyone. Please, feel free to comment on the doc or in this issue. Also, I'll be attending the next WECG meetings for further discussion.
Looking forward to working together
Does the current proposal imply that a user script will only run on a page?
What if a user script needs to run in a sandbox unrelated to a page?
I started an API proposal doc
Methods that contain Content
are already misleading because they also deal with main-world scripts that aren't content scripts by definition, but page
scripts: #313. If you want to separate methods by areas of concern then there should be Page
methods e.g. registerPageScripts and so on, thus obviating the need for world
parameter in Content and Page methods.
The main problem is that this API proposal is currently unusable in Violentmonkey or Tampermonkey because it's completely insecure due to https://crbug.com/1261964, to circumvent which these extensions currently have to extract the safe methods from a temporary iframe before running the main world scripts. It means that the main world script cannot call any standard method like addEventListener until the extracted methods are passed securely from this iframe in a manner that cannot be intercepted/spoofed by another main-world script.
Unless this bug is fixed or a secure communication method is provided, these extensions will become wide-open backdoors and I will suggest their authors to remove them from the web store. The method must be synchronous, by the way.
Another nasty problem is that the API proposal doesn't support @include
and @exclude
which are glob-based by default but can be RegExp as well, which is used by tons of userscripts and there's no good alternative for RegExp. With the current API shape any userscript with a RegExp include/exclude that runs at document_start
will have to run on all sites and then the extension's content script will perform the check, which is extremely wasteful (due to the need to inject the script into every web page + the lack of code compilation cache for it) and can take a long time depending on the regexp.
@tophf: I made some suggestion changes to the document. They haven't been approved but I think it's worthwhile for you to take a look at them and check if they suffice.
Does the current proposal imply that a user script will only run on a page? What if a user script needs to run in a sandbox unrelated to a page?
User scripts will only be allowed to execute in the MAIN world (page) or a new USER_SCRIPT world.
Is there some way to provide additional functionality (GM.xhr and friends) to a user script? For example the Firefox userScripts.onBeforeScript
let you run a callback to export functions into the execution environment. The current proposal has no such thing and thus the best alternative is to concatenate some fixed string of functions to the "code" argument?
Is there some way to provide additional functionality (GM.xhr and friends) to a user script?
@Sxderp It's there. Just not much fanfare calling to it:
scripts targeting ISOLATED are executed before USERSCRIPT. This allows the extension to prepare the execution environment before the user script code is executed.
This is inside "Other considerations" sub-chapter
I'll be attending the next WECG meetings for further discussion.
I would suggest holding a separate topic-specific meeting rather than discussing it in a regular meeting. Browser vendors and scripts manager maintainers are invited to participate. (PS: I am not script manager author)
Having different types and methods than content scripts is beneficial in the following ways: β¦β¦
I recommend using a completely different namespace(e.g. chrome.userscripts.register()
) so that they are in completely different document pages and developers don't get confused.
My other thoughts
What I understand is this proposal still allows arbitrary local and remote code to be executed in the main or userscript world of web pages, just don't allow to use extension api.
Besides extension and theme, can the web store support a third type of user script? All user scripts are submitted to the store and reviewed? Thus, user scripts are trustworthy and deterministic like extensions.
Compared to user scripts, I think content scripts should also allow arbitrary code execution. Because, when websites changed their web pages's structure/code/style, content scripts also need to be updated quickly to adapt to changes.(Usually, extenson developers do not own the site)
Besides extension and theme, can the web store support a third type of user script? All user scripts are submitted to the store and reviewed? Thus, user scripts are trustworthy and deterministic like extensions.
Not a good idea, one of the main benefits of userscripts is less oversight and you can self host them so updates or changes don't require long review processes. Also, most used userscripts that deal with YouTube, Spotify etc will never be allowed on chrome store.
Compared to user scripts, I think content scripts should also allow arbitrary code execution. Because, when websites changed their web pages's structure/code/style, content scripts also need to be updated quickly to adapt to changes.(Usually, extenson developers do not own the site)
This is not necessary, there are ways to observe changes on the page for example mutation observers.
What I understand is this proposal still allows arbitrary local and remote code to be executed in the main or userscript world of web pages, just don't allow to use extension API.
The extension needs additional permission for it. Thus the market can warn users that it's a potentially dangerous extension, also, reviewers will check if it's a valid usage of usercripts, like e.g. a weather app doesn't need it and it will very suspicious if it asks for it.
Besides extension and theme, can the web store support a third type of user script?
No. Sometimes you need US only for yourself, and sometimes even for a short period of time. Also, if a US need only a very limited number of people (<10), there is no reason to "publish" it.
Also, there are in times, in dozens more US than extensions, and Google won't be glad to spend money on reviewing every single US.
Compared to user scripts, I think content scripts should also allow arbitrary code execution.
Enforcing security is one of the main goals in the MV3, and forbidding arbitrary code execution is one of the main ways to achieve it.
Because, when websites changed their web pages's structure/code/style, content scripts also need to be updated quickly to adapt to changes.
Sites do not roll up changes every single day, most of them even prefer to avoid this (it requires spending money and receiving usually angry feedback that things anymore aren't what the user used to, unless it's bug fixing). Also, you can in an empirical way find general and fuzzy approaches that are more or less resistant to minor changes in the page layout.
No. Sometimes you need US only for yourself, and sometimes even for a short period of time. Also, if a US need only a very limited number of people (<10), there is no reason to "publish" it.
I understand. In my previous comment, I said "script manager can supports run both trusted and untrusted code, but gives different user warnings.". In other words, no matter it is published to the store or not, users can run them.
Also, there are in times, in dozens more US than extensions, and Google won't be glad to spend money on reviewing every single US.
This question is best answered by the extension stores. I don't know what they think.
Enforcing security is one of the main goals in the MV3, and forbidding arbitrary code execution is one of the main ways to achieve it.
In this way, I think user scripts should be subject to the same restrictions as content scripts.
Sites do not roll up changes every single day, most of them even prefer to avoid thisβ¦β¦
I often see extension developers complaining that a website update caused its code to go wrong, and they need to update the extension immediately.
Methods that contain
Content
are already misleading because they also deal with main-world scripts that aren't content scripts by definition, butpage
scripts: #313. If you want to separate methods by areas of concern then there should bePage
methods e.g. registerPageScripts and so on, thus obviating the need forworld
parameter in Content and Page methods.
First off, chrome.scripting.getRegisteredContentScripts
should have been chrome.scripting.getRegisteredUserScripts
. My bad, fixed it.
As for the separate methods, correct me if I am wrong, you are suggesting having separate methods for user scripts, content scripts and page scripts (basically dividing content and page scripts) or either changing the current content script methods (issue #313). Content and page scripts behave similarly, have the same permissions enforcement and most importantly they don't support remote host code. Thus they can be grouped under the same methods (as they are currently) with different worlds. User scripts have to be clearly differentiated for better documentation and stronger enforcement of remote host code abuse. Thus the introduction of new methods. However, as @hanguokai points out, maybe is better having a new namespace.
As for your rename suggestion, I see the issue is listed as an Agenda discussion for public meeting on 2022-11-10 so better to follow up there.
The main problem is that this API proposal is currently unusable in Violentmonkey or Tampermonkey because it's completely insecure due to https://crbug.com/1261964, to circumvent which these extensions currently have to extract the safe methods from a temporary iframe before running the main world scripts. It means that the main world script cannot call any standard method like addEventListener until the extracted methods are passed securely from this iframe in a manner that cannot be intercepted/spoofed by another main-world script.
Unless this bug is fixed or a secure communication method is provided, these extensions will become wide-open backdoors and I will suggest their authors to remove them from the web store. The method must be synchronous, by the way.
I was not aware of this issue. I'll contact the OWNER and see what's the status on this. Thanks for pointing it out
Another nasty problem is that the API proposal doesn't support
@include
and@exclude
which are glob-based by default but can be RegExp as well, which is used by tons of userscripts and there's no good alternative for RegExp. With the current API shape any userscript with a RegExp include/exclude that runs atdocument_start
will have to run on all sites and then the extension's content script will perform the check, which is extremely wasteful (due to the need to inject the script into every web page + the lack of code compilation cache for it) and can take a long time depending on the regexp.
How are RegExp currently supported in MV2? I believe chrome.tabs.executeScript
doesn't support @include
and @exclude
. If they are not currently supported, they may not be included on the API v1 since we are trying to achieve functional parity with MV2. However, do you have any suggestions on how this can be achieved?
Sorry for the late response everyone. I am getting myself familiar with user scripts and user scripts managers to provide better answers. Will continue shortly.
Content and page scripts behave similarly, have the same permissions enforcement and most importantly they don't support remote host code. Thus they can be grouped under the same methods (as they are currently) with different worlds
As I pointed out in #313 they are so different that you could say they're "worlds apart", pun intended, so conflating them is a very dangerous path that misleads developers into believing their data and code are safe from interception/spoofing/poisoning.
How are RegExp currently supported in MV2?
There's no support at all. The closest thing is RE2 syntax in filtered events of declarativeContent or webNavigation API.
Currently userscript managers explicitly use RegExp
matching in JS. It doesn't cost much in ManifestV2 because the regexps are compiled just once in the persistent background script. Doing the same in a ManifestV3 service worker will be very costly due to the need to recompile regexps every time the SW starts for a new URL navigation, and it won't guarantee document_start timing.
I understand that Chrome won't add full RegExp to its API, so the next best thing is implementing RE2-compatible matchesRegex
and excludeMatchesRegex
and a method to test compatibility similar to chrome.declarativeNetRequest.isRegexSupported, so that userscript managers can simplify the userscript's regex for the API and then narrow it down inside the content script using the full RegExp check, thus reducing the wasteful injections.
@EmiliaPaz Thanks for sharing your proposal! I'll share my feedback here, because it's difficult to trace or reference discussions on Google docs, especially on a document that's still undergoing revisions.
Your stated goal of the document is "Achieve functional parity with MV2 for user script managers.". I note that just MV2 parity is a low bar. Due to the lack of supporting APIs from the browser, user script managers are built insecurely. Improving that would be great.
I'll describe the needs of user script managers in more detail, particularly focused on security and feasibility. After that I'll summarize the requirements as a starting point for the API design. I am intentionally focusing on parts that cannot be implemented by user script managers themselves, as these should be the top priority. E.g. regex matching can be implemented by user script manager code themselves, and support for that could be a future enhancement.
Background
There are many user script managers (USM hereafter), with the most popular ones being Greasemonkey, Tampermonkey and Violentmonkey. This section describes how they work, and the core primitives that they rely on today. Other than the "userscript" world, all concepts mentioned here already exist in MV2 today.
User scripts run on top of existing web pages, and need to access the DOM and/or JS variables in the page's context and/or semi-privileged user script APIs:
- https://wiki.greasespot.net/Greasemonkey_Manual:API
- https://www.tampermonkey.net/documentation.php
- https://violentmonkey.github.io/api/gm/
The following questions are relevant for the choice of world for a user script:
- Does the user script need access to variables in the web page? (e.g. via
unsafeWindow
). - Does the user script need access to semi-privileged user script APIs (e.g.
GM
)?
need page access | need GM access | Ideally, the world to run in | World used by MV2 USM |
---|---|---|---|
no | no | User script world | Main world or isolated world (of content script) |
no | yes | User script world, with prepared environment | Main world or isolated world (of content script) |
yes | no | Main world | Main world |
yes | yes | Main world (or user script world + supporting code in main world) | Main world |
When semi-privileged APIs are not needed, a USM only needs the ability to run some given string as JavaScript code. Ideally in a world distinct from the page/content script, i.e. the "userscript" world proposed in #279 (comment).
When semi-privileged APIs are needed, but page access is not, likewise (with some extra work from the USM to expose the APIs).
document_start
), that saves all necessary (DOM/JS) API references in local variables, under the assumption that the web page has not had a chance to modify the execution environment. This has limited success, because the assumption is not always valid (e.g. https://crbug.com/1261964 mentioned by #279 (comment)).
This is generally the structure of any script that introduces semi-privileged functionality to the main world. It itself runs in the main world (the concept is also relevant outside user scripts):
(function(secrets) {
// Step 1: Save a reference to all standard DOM API methods, to avoid tampering by the web page.
// Step 2: Use |secrets| and DOM APIs to create a semi-secure communication channel, e.g. using `CustomEvent`.
// Step 3: Via this semi-secure channel, retrieve necessary data such as user script metadata.
// Step 4: For *each* user script, create the GM API interface and execute the user script:
somehowExecuteUserScript(localVarWithGMAPI, window, etc);
})("secret communication ID here");
Ordinarily, web pages and also scripts in the main world can use window.eval
to execute code. This can however not be relied upon because it may be blocked by the page's CSP (script-src without 'unsafe-eval'
). The actual content script code is therefore executed by inserting an inline <script>
element containing the definition of a user script (this is blocked in MV3 due to the CSP for content scripts that blocks remote code):
window.someSecretRandomName = function(GM, unsafeWindow, etc) {
/* actual user script code here */
};
The user script is wrapped in a function, because it needs a closure with a reference to the semi-privileged API, which it receives when called by the previous script that starts with (function(secrets) {
.
Identified requirements
The following required primitives can be derived from the above overview of how user script managers work:
- A mechanism to execute code (user script or USM API definition) in the main world.
- The ability to execute code in a world different from the main world (current practice is the "isolated" world, but ideally the "userscript" world proposed in #279 (comment)).
- A mechanism for that code to receive a set of private well-known safe values/API references that cannot be tampered with by the execution environment of the main world (resilient against prototype pollution).
- A mechanism for the user script manager to selectively expose APIs to user scripts, potentially distinct for each user script.
@EmiliaPaz 's proposal in #279 (comment) covers the first two aspects, but not the other two.
There are many different ways to implement these requirements. I will show two examples, one userscript-specific API (from Firefox) and a more generic API that has broader applicability.
Final note: the above requirements are minimal. If the goal is to move the responsibility of matching and triggering script execution from the extension to the browser, then there should also be a way to register some metadata for a script that is synchronously available. It would also be nice if there is a way for content script to veto the execution of a user script. These can be supported in the two API designs below.
Firefox's userScripts API (MV2)
Firefox's userScripts API was mentioned before in #279 (comment). I'll briefly describe the relevant parts of the API design for inspiration.
Features of the userScripts
API:
- Each user script is executed in its own isolated execution environment.
userScripts.register
method to register user script code, including the ability to define metadata for theapi_script
.- Optional
user_scripts.api_script
manifest key to specify a script that is expected to prepare the execution environment of the user script. - api_script's
userScripts.onBeforeScript
event to notify the api_script before a user script is executed, to prepare the execution context of the environment. It receives the metadata from the script registration, to allow the USM to selectively expose APIs via thedefineGlobals
method.
In Firefox, it is possible to have asymmetric access relationships between code from different execution environments (Xray Vision). This mechanism enables user script managers in Firefox to prepare the environment beyond what's possible with defineGlobals
, by assigning values directly to the execution context of the user script. User scripts cannot read/write values in the opposite direction; user script managers would have to use the export
API passed to the onBeforeScript
method to export the value to the user script.
The api_script
from this API is created once per process as an optimization. It is not strictly necessary, so if desired this characteristic can be dropped. The onBeforeScript
event can be added to a content script instead.
Note: we have marked the userScripts API as MV2-only in Firefox because the script registration mechanism is incompatible with the lifetime characteristics of background scripts in MV3. Furthermore, we would like to establish a cross-browser-compatible API for user script managers.
Scripting API with cross-world communication
To avoid confusion, this example extends the current scripting
API, not the updateUserScripts
API sketched in @EmiliaPaz's proposal.
- The
scripting.registerContentScripts
method receives two new options:code
- code to run in the main or isolated worldextraArgs
- list of parameters that should be resolved by the content script. This cannot be used if the world is "ISOLATED". Note that there is already anargs
option in the API; that could be re-used to support static metadata.
chrome.scriptingContent.onRequestArg
- a new event in the content script (isolated world) to allow the content script to respond to requests for arguments.- I named this "Content" because this API namespace is part of scripts running in the (web) content process, opposed to the extension process.
// Somewhere in the extension, e.g. extension page where the user edits the user script.
chrome.scripting.registerUserScripts({
code: `
// This is what the user script manager generated:
let GM = {
info: GM_info,
};
let unsafeWindow = window; // In the main world, unsafeWindow is the window.
let someRandomNumber = args[0];
args = undefined;
// This is the actual user script:
console.log(hello, GM.info(), unsafeWindow);
// ^ note: console is a global variable. The web page could have tampered with it...
`,
externalArgs: ["hello", "GM_info"],
args: [4], // <-- someRandomNumber
id: "my-userscript",
world: "MAIN",
matches: ["https://example.com/*"],
});
Internally, the browser could convert the above to the following:
function(args, hello, GM_info) {
/* the code string here */
} // TODO: browser should find the hello & GM_info args and call this function
Then, before the browser runs the user script, it asks the content script for the API definition. For example:
chrome.scriptingContent.onArgRequest.addListener(arg => {
if (arg === "hello") {
// We must at least support primitive values (string, number, etc).
return "hello world";
}
if (arg === "GM_info") {
// We must also support functions. Potentially async (i.e. returning a Promise).
return async () => {
// The functions themselves may return values. TODO: define acceptable return values.
return { plain: "object", is: "safe", but: "what", about: Function ? "callable properties" : "?" };
};
}
});
To maximize utility, I suggest to support at least the following return types:
- Functions. The functions themselves should return a value that matches this format.
- DOM elements, anything for which a meaningful DOM wrapper exists across worlds.
- Anything else that can be structurally cloned.
The above description consists of standard types that are well-supported across browsers. Objects with getters, setters or function properties are not supported.
Note: the above is intentionally generic to cover broader use cases such as #284.
Is there some way to provide additional functionality (GM.xhr and friends) to a user script? For example the Firefox
userScripts.onBeforeScript
let you run a callback to export functions into the execution environment. The current proposal has no such thing and thus the best alternative is to concatenate some fixed string of functions to the "code" argument?
Is XHR necessary or can it be shimmed using fetch? Key constraint is XHR supports blocking requests.
Another nasty problem is that the API proposal doesn't support
@include
and@exclude
which are glob-based by default but can be RegExp as well, which is used by tons of userscripts and there's no good alternative for RegExp. With the current API shape any userscript with a RegExp include/exclude that runs atdocument_start
will have to run on all sites and then the extension's content script will perform the check, which is extremely wasteful (due to the need to inject the script into every web page + the lack of code compilation cache for it) and can take a long time depending on the regexp.
Do you, or someone else, know how close do we get with:
- Just
matches
(excludeMatches
) matches
+globs
matches
+ regexmatches
,globs
, and regex
Basically wondering how much we lose without regex support, and if supporting globs would make a meaningful dent.
EDIT: I just realized @Rob--W commented, and included regex. Will go over it and comment back
Is XHR necessary or can it be shimmed using fetch? Key constraint is XHR supports blocking requests.
Clarification: it's GMXHR i.e. GM_xmlhttpRequest (GM.xmlHttpRequest), not XMLHttpRequest.
GMXHR cannot be polyfilled (inside the web page process) because it's cross-origin, has access to the Forbidden HTTP headers, all cookies, all response headers including Set-Cookie, and runs in extension background context.
GMXHR does not support synchronous operation in modern USM, so it's not a concern.
In Tampermonkey/Violentmonkey GMXHR uses XMLHttpRequest as the backend, but it's possible to switch to fetch
mode (already implemented in Tampermonkey) at the cost of losing onprogress events or significantly complicating its implementation (this is a known problem of the Fetch API design not unique to USM).
Just matches (excludeMatches)
Covers 50% or less of userscripts, I guess. This is the current shape.
matches + globs
Covers 95% of userscripts, I guess.
matches + regex
Covers 55% or less of userscripts (50% from 1 + 5% estimate for regex from 2)
matches, globs, and regex
Covers 100%, assuming it's the full JS RegExp. For RE2 see my comment above.
These numbers are in the correct order of magnitude.
From the userscript.zone database, which lists the most popular publicly available scripts:
Just matches
59,7%
matches + regex
64,2%
matches + globs
95,1%
only regex (for reference)
4,9%
Note: userscript.zone uses RE2 because I had strange out of memory issues with RegExp quite often and IIRC only a few scripts didn't work with RE2 syntax. I can find out the real numbers if necessary.
Important implementation note given the above discussion on matches + globs.
Just declaring matches
and globs
together in the userscript definition and then re-using the same internals that the browsers already have for content scripts does NOT result in a usable API for user script managers.
- User script managers expect matches (
@matches
) and globs (@include
) to work as a logical disjunction (OR) - the script runs if either is matched. - Chrome (and Firefox)'s content script APIs currently treat
matches
and globs as a logical conjunction (AND) - the script runs only if both are matched.
In my opinion, globs can easily be translated to RegEx. So, just having a single RegEx input for includes and a single RegEx input for excludes (exceptions to the include) should be enough to implement all cases.
Ofc, if there's more options, even better!
globs can easily be translated to RegEx
Yes, but that consumes a lot more memory and CPU, so it's not a good thing for ManifestV3 which is also about performance.
The largest use of CPU with "decently made" Regex is compilation. I think the only exception is explosive Regex.
Important implementation note given the above discussion on matches + globs. Just declaring
matches
andglobs
together in the userscript definition and then re-using the same internals that the browsers already have for content scripts does NOT result in a usable API for user script managers.* User script managers expect matches (`@matches`) and globs (`@include`) to work as a logical disjunction (OR) - the script runs if either is matched. * Chrome (and Firefox)'s [content script APIs](https://developer.chrome.com/docs/extensions/mv3/content_scripts/#matchAndGlob) currently treat `matches` and globs as a logical conjunction (AND) - the script runs only if both are matched.
Indeed, that is an important consideration when using a register type API. The issues has been experienced in FireMonkey which AFAIK is had been the only USM using the dedicated userScripts API.
In order to support Glob @include
, FM has to manually add a catch-all matches
(i.e. matches: ['*://*/*', 'file:///*']
) which can result in unexpected outcome when there are mixed @matches/@exclude-matches
& @include/@exclude
in a userscript registration.
[...] which AFAIK is the only USM using the dedicated userScripts API
FWIW: Tampermonkey lately uses the userScripts API as content script replacement to allow userscripts to overcome (all remaining) CSP issues without executing them in the content script world and to allow userscripts to run in a clean execution environment if wanted. TM also uses <all_urls>
to support @include
.
In my opinion, globs can easily be translated to RegEx.
As well as matches. :-)
FWIW: Tampermonkey lately uses the userScripts API as content script replacement to allow userscripts to overcome (all remaining)
I stand corrected... I was not aware :)
Another userscript feature that's going to be broken by ManifestV3 is GM_addElement, which allows userscripts to add a script
, style
, link
elements bypassing the CSP of the page to add their own external stylesheets or additional code that needs to run in the MAIN world (to monkeypatch some web platform API) when the userscript itself runs in an isolated world.
It works in Chrome (not in Firefox) because this is how Chrome's ManifestV2 CSP for content scripts works. It doesn't work like this in ManifestV3, so the users will have to disable CSP of the page entirely, thus reducing the security, because there's no way for a ManifestV3 extension to modify an existing CSP and just add a nonce
exception for itself - neither via the HTTP header in declarativeNetRequest nor the HTML-embedded <meta http-equiv="Content-Security-Policy" content="...">
equivalent.
Another userscript feature that's going to be broken by ManifestV3 is GM_addElement, which allows userscripts to add a
script
,style
,link
elements bypassing the CSP of the page to add their own external stylesheets or additional code that needs to run in the MAIN world (to monkeypatch some web platform API) when the userscript itself runs in an isolated world.It works in Chrome (not in Firefox) because this is how Chrome's ManifestV2 CSP for content scripts works. It doesn't work like this in ManifestV3, so the users will have to disable CSP of the page entirely, thus reducing the security, because there's no way for a ManifestV3 extension to modify an existing CSP and just add a
nonce
exception for itself - neither via the HTTP header in declarativeNetRequest nor the HTML-embedded<meta http-equiv="Content-Security-Policy" content="...">
equivalent.
Wouldn't it be better is userscripts were able to bypass the page's CSP anyway (possibly requiring a special 'GM_BypassCSP")?
Here is a simple way of looking at the userscript environment.
Normally, there are 3 distinct JavaScript contexts in an extension:
Privileged
: where the script (e.g. options, popup) can interact freely with browser API (subject to permissions)Content
: where the Content scripts have limited interaction with browser API (i.e. WebExtension APIs), plus interaction withpage
contexts (andpage
DOM)Page
: the webpage JavaScript
There are already safeguards in place to govern above.
Userscripts require an additional context e.g. userscript
Userscript
context is similar tocontent
context but has NO browser API access- Like
content
context userscript context can interact withpage
context (andpage
DOM), but NOT vice versa - Unlike
content
context where multiple scripts would share the same context, multipleuserscript
contexts are required, one for each userscript and each one must be isolated from the other - A special
content
context (with limited browser API) is required for auxiliary functions (i.e. GM functions) for eachuserscript
context and a communication method between this special context and the corresponding userscript context
Wouldn't it be better is userscripts were able to bypass the page's CSP anyway (possibly requiring a special 'GM_BypassCSP")?
Userscripts should just be able to transparently ignore CSP in order for them to run
Security: Userscript and CSP/CORS
There are a number of considerations for userscripts when dealing with page CSP & CORS.
CORS
- Userscripts could get the same CORS rules as
content
scripts
CSP
Sites setting strict CSP rules:
- For legitimate security (e.g. banks, or other sites dealing with sensitive data)
- In order to prevent user-alternation (e.g. remove ads)
- I have seen sites that even crash console intentionally if user attempts to open browser console
- There are many sites that attack (e.g. high CPU/RAM) users who don't want their intrusive ads
- For no apparent reason
Injection by Userscript
- CSS: Userscripts should be able to inject CSS (as it is low risk)
- Non-executable DOM: There are pros/cons arguments (although not high risk)
- Executable: Requires further discussion
Footnote
Traditionally, GM XHR/fetch is performed from the background script where the request is sent without any origin. Bearing in mind the removal of XMLHttpRequest
in background service worker in Chrome in MV3 (ref: Cross-origin XMLHttpRequest), if the apiScript
(not the userscript) could send XHR/fetch without origin, the dependency on the background script could be reduced and most GM functions could be processed in the apiScript
.
In an effort to help focus the discussion, I'd ask that member share feedback in GitHub issues rather than in linked docs. Generally speaking docs are a point-in-time reference, not canonical resources. As such, comments in Google Docs should be limited to minor clarifying questions and suggestions should be limited to typos and other cleanup changes.
Moving forward, I will be closing out doc comment threads and suggestions that are too large to be handled in the doc. Where appropriate, I will ask the contributor to move the comment over to GitHub issues. In a few moments I will start close out most comments in Emilia's API proposal.
@brunoais, you have a number of comments and suggestions in Emilia's proposal that may be better suited for longer discussion here or an new doc to iterate on this proposal. As I mentioned, I'm going to close out comments to keep Emilia's proposal more focused. If you wish to reference these comments/suggestions, I created a copy of the doc with all comments/suggestions before I started closing them out.
@dotproto
I can't find my change suggestions to move them here. I also can't find them in the clone document you did. Where can I find the suggestions I made so I can place them here for discussion?
Friendly reminder: Today, there is special WECG meeting for User Scripts.
15 November 2022, 08:00 - 08:55 America/Los Angeles.
Whether it is useful or not, I would like to describe the positions of all parties:
- Website Owners: I don't like third-party injected code, is there any way to limit them?
- User Script Developers: I want as few restrictions as possible, such as arbitrary code and dynamic updates.
- User Script Manager: I'm just providing an injection mechanism, and I don't take any responsibility for these code.
- Extension Developers: I have to comply with a wide variety of restrictions and requirements from the extension store, but user scripts don't have any restrictions.
- Browser Vendors and Extension Stores: I need to meet the needs of the user, while ensuring security and privacy. I take some responsibility for malicious extensions. As for malicious user scripts, it should be the responsibility of the users themselves.
- Website Owners: I don't like third-party injected code, is there any way to limit them?
Interesting... As a website owner, I actually welcome them. One of the main things I can welcome is more features and less work for me and users can just choose what extra features they want, at their risk, by installing the userscripts they want.
Are website owner complaints common enough to cause that?
People often mistake extension code with user code.
- Extension code is what the developer has decided to run
- The code that is packed with the extension
- No remote code chosen by the developer is allowed
- User code is what the user has decided to run
- User pasting code into the Developer Tool
- User selecting a userscript to run
An extension is not allowed to select a remote userscript to run, however users are allowed to.
The difference is who chooses the script to run.
USM simply facilitates users to run scripts chosen by the users.
An installed USM does nothing, runs no remote code, even if left forever.
It is the user that instructs the USM to run the script selected by the user. It is purely users' choice which must be honoured.
Footnote
While most of the discussion has been about userScripts, userStyles/userCSS are also part of this discussion.
@erosman Your comment doesn't conflict with what I said. I didn't say USM is not extension.
@erosman I clearly distinguished between user script and user script manager. In that statement, I said "user script", not "user script manager". User scripts are written by user script developers(usually not users themself).
@hanguokai You are right. I misread. I thought you were referring to the userscript managers. Sorry about that. π€¦ββοΈ
(I removed the post that was made in error to tidy-up the topic.)
I have heard the "USM vs other extensions" argument so many times that my mind went straight to it.
Back to the subject in the comment..... user scripts
- Userscripts are not hosted on extension stores
- Userscripts are not governed by the browsers, the same way code pasted to the console is not governed by the browsers
Therefore, any such comparison between an extension & a userscript is irrelevant.
Please note:
- Users are in favour of being able to run supplementary scripts of their choice; as evident by 10m+ users of userscript/userstyle managers
- Browsers & USM provide an environment that is as safe as possible for the users to run userscripts
- Userscript managers endeavour to cater for the requirements of the userscript developers as much as possible, without sacrificing user security or privacy (please note #279 (comment))
Therefore, any such comparison between an extension & a userscript is irrelevant.
When the extension meets the following conditions, I think user script and extension are the same:
- Extension only uses content script feature
- Extension opens source code to the user
- Extension explicitly tells the user when/how the code run
- After the user has read above information, and the user chooses to install the extension.
The extension-userscript relationship, is similar to the browser-extension relationship.
- Browsers have access to areas that extensions do not
- Browsers have powers that extensions do not
- Extensions have access to areas that userscripts do not
- Extensions have powers that userscripts do not
Extensions have a subset of capabilities of the browsers. Userscripts also have a subset of capabilities of the extensions.
- Extension only uses content script feature
Extension can inject script into content, page & more. Extension can do whatever a userscript does and more.
- Extension opens source code to the user
Userscript source code is also available to the users.
Furthermore, not all extensions are open source.
- Extension explicitly tells the user when/how the code run
Ideally, true but not 100% in practice. There are many aspects of the extensions that are not evident due to practicality or other reasons.
GA is a prime example in many extensions. How many extensions that use GA advertise its use?
- After the user has read above information, and the user chooses to install the extension.
The same applies to userscripts. It is extremely rare that a user installs a userscript without having read what it does.
π Nonetheless, installing a userscript, good or bad, is users' choice, and users' choice must be respected.
Furthermore, as mentioned earlier, userscripts are not regulated by the browser or extension stores. All browser and extension developers can do is to provide an as-safe-as-possible environment for it.
AFA I understand, the purpose of this topic is how best to provide the aforementioned environment.
Footnote
Even with the current restrictions in MV3, it is possible to create an extension that supports userscripts, albeit it would not be as safe as a properly thought of environment.
The above comment, I think you understand what I said, and this is what everyone here knows.
Look forward to seeing how it going! A balance should be struck between dev's innovation and user's safety
In today's WECG meeting, someone asked for a summary of the user scripts meeting. That meeting had its meeting notes merged in #324, now available at https://github.com/w3c/webextensions/blob/main/_minutes/2022-11-15-wecg-userscripts.md
The summary is as follows:
The original goal of the meeting was to review the goals and scope of the user script API design. Googleβs objective of this effort is to reach feature parity with MV2, and identify the foundations needed for future enhancements.
The initial API suggestion shared by Simeon was to add a βcodeβ parameter to the scripting.registerContentScripts API to allow execution of strings. I recommended at least introducing a new userscript βworldβ for security reasons. But even with this, the work in progress API design would not address the needs of user script managers (background in this comment).
In the design of a user script API, there are roughly two approaches to take: let the browser manage the matching and injection, or let an extension run an all_urls content script that takes care of the injection. User script managers currently do the latter, and when asked, the Tampermonkey dev in the meeting expressed the desire to continue doing this. This reduces the scope of the API design.
The main blocker for user script managers in MV3 compared to MV2 is the inability to run strings as code in the main world (pageβs context). In MV2 an inline <script>
element offered this capability in Chrome, but MV3 has a CSP in content scripts that blocks this. The work-in-progress API fills the gap, not by the βcodeβ parameter, but due to not enforcing a CSP in the userscript world. Instead of working around the limitation via this accidental implementation detail, I proposed to focus on two API aspects: allowing a content script to run a string as code in the main world, and a mechanism for secure cross-world communication.
allowing a content script to run a string as code in the main world
FWIW, there's even a ticket for Chrome: https://crbug.com/1207006
In the design of a user script API, there are roughly two approaches to take: let the browser manage the matching and injection, or let an extension run an all_urls content script that takes care of the injection. User script managers currently do the latter, and when asked, the Tampermonkey dev in the meeting expressed the desire to continue doing this. This reduces the scope of the API design.
Am I misinterpreting or does this mean that the new API goal is to please a minute number of developers who happen to be the authors of a specific type of extension?
What about extensions that let the user run scripts at any given moment? That is, not at page load, or page idle, but at any other moment the user desires to do so.
For example, will it be possible to create an extension that allows the user to run a given script in a given tab immediately? i.e. not in the future, but right now at this very moment when a page is already loaded in the given tab.
Extensions that allow users to automate tasks such as filling form fields or clicking on buttons depend on that functionality.
Extensions that allow users to automate tasks such as filling form fields or clicking on buttons depend on that functionality.
Can't you do that with only premade code? With the user just mentioning what he wants to do and you run functions you already made?
Or do you mean actual custom code?
Am I misinterpreting
Apparently you are. The desire is to retain the control of an extension over how and when the injection/matching takes place.
Can't you do that with only premade code? With the user just mentioning what he wants to do and you run functions you already made? Or do you mean actual custom code?
Custom code, of course. Snippets of Javascript provided by the user.
The desire is to retain the control of an extension over how and when the injection/matching takes place.
That's exactly what I meant with "specific type of extension". The "matches" parameter determines the URLs where the script must be injected and the "runAt" parameter has only 3 possibilities.
Which means this API proposal only attempts to support user script managers such as Grease Monkey and similar.
The use case I described in my previous comment won't be possible.
The quote is rather vague. All it says is "continue doing this", so I assume "this" means everything userscript extensions are currently doing and that includes arbitrary execution of code via executeScript which Tampermonkey needs anyway for its @run-at context-menu
.
Regardless of these particulars, a "userscript extension" means any extension which is designed to run user-provided code, not just userscript managers. Apparently this is yet another vague term that might need a better substitute.
Initially, MV3 did not allow for injection of code strings. The purpose of this topic is to discuss how user-code can be injected by an extension as safely as possible.
What about extensions that let the user run scripts at any given moment?
For example, will it be possible to create an extension that allows the user to run a given script in a given tab immediately?
Custom code, of course. Snippets of Javascript provided by the user.
Sure....
- MV2: tabs.executeScript()
- MV3 (after this proposal is implemented): scripting.executeScript()
That's exactly what I meant with "specific type of extension". The "matches" parameter determines the URLs where the script must be injected and the "runAt" parameter has only 3 possibilities.
That is a specific registration of JS/CSS for auto-injection, which is available to any extension and used by many non-userscript-manager extensions.
- MV2: contentScripts.register() & userScripts.register()
- MV3 (after this proposal is implemented): scripting.registerContentScripts()
@erosman, do you mean that scripting.executeScript()
will also support a code
parameter?
I couldn't find that in the API proposal document.
@erosman, do you mean that
scripting.executeScript()
will also support acode
parameter? I couldn't find that in the API proposal document.
It is also a part of the proposal to support the injection of code
parameter or similar in scripting.executeScript()
.
The important part is to finalise where (which context) the user-code will be injected into.
Semi-privileged content
context, which is meant for extension's own code, is not best suited for user-script/user-code.
Proposal, that takes into account multiple points discussed here, was moved to PR #331 to facilitate discussion
Just want to point out, that there are other use cases for injecting user scripts than just by match patterns.
For example mouse gesture extensions want to inject them into a certain tab (active tab) at a certain point (at the end of a a gesture).
Example add-ons with code/implementation reference:
Therefore I plead to have a code
parameter for the scripting.executeScript()
function as well.
This is implemented in Chrome. Future expansions will be tracked separately (you can see some of these in @Rob--W 's comment at #279 (comment)). We'll leave this issue open until other browsers implement the core API, and then close this out.
Firefox 136 has enabled support for the userScripts API, including support for multiple worlds (#565).
Documentation is available at https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/userScripts
A code sample is available at https://github.com/mdn/webextensions-examples/tree/main/userScripts-mv3
The feature is not available on Firefox for Android yet, follow https://bugzilla.mozilla.org/show_bug.cgi?id=1949955 to see its availability.
userScripts.execute
(#477) has not been implemented yet either, implementation progress on that can be followed at https://bugzilla.mozilla.org/show_bug.cgi?id=1930776
A bit unfortunate how userScripts
works out. I really liked the onBeforeScript that was available in the Mozilla vendor specific API. It allowed you to export some symbols to the user script while still running within the privileged context of the extension. I'm sad to see there isn't an equivalent in the new API.
It seems the best we have is how it was before Mozilla's vendor specific API. That is, you inject / append some code to the top of the user script itself. Granted, the interface is a little nicer since you can include entire files rather than manually concatenating, but it's still a major blow.
The exportFunction
and cloneInto
functions were also very convenient when you had to significantly munged the target page. Those functions were generally provided via the onBeforeScript listener. See firemonkey as an example. I haven't done any testing so I can't say for certain, but it seems like they have been lost in the new implementation.