Support variables when resolving values in settings
OlsonDev opened this issue · 203 comments
Hi,
I was just reading the latest updates and it says one can install typescript@next globally and then set typescript.tsdk so VS Code can use the appropriate version/installation. In a team environment, I'd like to put that setting in our project, something like:
.vscode/settings.json:
{
"typescript.tsdk": "%APPDATA%/npm/node_modules/typescript/lib"
}The problem is restarting VS Code results in an error:
The path c:\Projects\Derp\%APPDATA%\npm\node_modules\typescript\lib doesn't point to a valid tsserver install. TypeScript language features will be disabled.
Now the setting needs to be per-person because I highly doubt my teammates have tsc installed in C:\Users\Olson\AppData\Roaming\npm\node_modules\typescript\lib 😉
Could we get environment variables evaluated on that and all other settings that involve paths?
I haven't tested other paths, but I see these in the Default Settings:
git.pathmarkdown.stylesjson.schemastypescript.tsdkphp.validate.executablePath
I was looking into this a little, but it'd be helpful to understand what the behavior should look like on non windows systems. Mainly, should we use the same environment variable %SYNTAX% or try to emulate the host platform?
I'd argue that having VS Code be consistent across multiple systems is more important than constancy with the host platform, but I'm curious to see if anyone has any objections to that. We could use $SYNTAX or something else, across all platforms too.
@mjbvz I thought about this when I first opened the issue but neglected to mention it because I'm indifferent. However, I do agree consistency within VS Code is likely more beneficial than with the OS.
Visual Studio seems to use $(PATH) syntax. I think that would make the implementation easier because, ya know, having an end delimiter tells you exactly what to look up instead of having to filter down Object.keys(process.env) one character at a time.
I guess my vote goes toward $(PATH) syntax for consistency within Microsoft IDEs. 👍
Just looking through the code, this best approach seems to be using the existing SystemVariables (or extracting its functionality) class since SystemVariables::resolve takes a string and substitutes in environment variables for any ${VAR} occurrences (see AbstractSystemVariables::__resolveString).
Another approach may be to update newConfigFile() from platform/configuration/model.ts to resolve the environment variables when setting the config values, but this would not pick up changes to environment variables after the config is saved.
Another approach may be to update newConfigFile from platform/configuration/common.ts to resolve the environment variables when setting the config values.
You mean platform/configuration/common/model.ts, I assume?
Yes, thanks. I've updated the comment with the correct file name.
Resolving the environment variables only when they are actually needed would be best, so that always pick up the latest values.
@DonJayamanne is starting to look into this from #8555
@pcgeek86, this capability hasn't been added yet.
The PR you mentioned was closed without merging the changes due to some other ongoing changes mentioned by @bpasero in his last comment
@DonJayamanne for your case (Python I think?), did you find a workaround to support this?
If we add variables support to settings, we need to think about validation. If a setting only allows for certain string values, putting in a variable would always result in warnings. I guess this problem already exists today for launch.json and task.json.
@bpasero, yes for Python extension i do have a work around, with support for $workspaceRoot and $env.. (env variables) in settings.json.
Here's an example:
"python.pythonPath":"${workspaceRoot}/env/bin/python"However, there's one problem.
The debugger can now reference settings defined in settings.json (via the following PR using the syntax ${config.python.pythonPath}).
Unfortunately the variable ${workspaceRoot} doesn't get resolved in the debugger. I cannot implement a work around either as the workspace Root path isn't available in the debugger either.
@bpasero , upon further analysis, I believe the bug is in configVariables.
The problem is the debugger is getting the resolved value from settings.json, however the resolved value needs to be resolved once again to get the tokens (such as ${workspaceRoot}) replaced. I'll raise a separate issue for that.
Great thanks 👍
@DonJayamanne did you put the right issue number in your comment?
Looking at this feature request, new configuration changes have nothing to do with this.
I was referring to #8555 (comment)
That was back in July/August.
So based on the changes since then, any suggestions?
For example, we now have the configurationResolverService which can be used to resolve variables like ${workspaceRoot}, ${file} etc.
Why not use this service to resolve the settings as well?
@sandy081 yes, you can use the configuration resolver service for resloving variables, as that is its prime duty. Another reason why this is not part of the configuration service is that we wanted to make all the methods in the configuration service sync and resolving has to be done async due to some potential variables which require input from the user (using quick pick for instance)
Thanks for the input. Agreed that we are not to change the semantics of Configuration service.
@ramya-rao-a Hope you got the answer for your question. So in short, Configurations are not yet supporting the variables and needs a proper implementation story to support it.
How about exposing this resolver service for extension authors? Then they can fetch settings first via vscode.workspace.getConfiguration and then use this service to resolve variables?
Furthermore, we can extend this same service to resolve env vars as well.
cc @jrieken
Another option perhaps: add a flag on setting definitions to determine if the stored value should be automatically resolved when retrieved. I imagine it could work like the "isExecutable" flag that we played around with in 1.9. Something like:
"configuration": {
"title": "Git",
"properties": {
"git.path": {
"type": "string",
"description": "%config.path%",
"default": "",
"resolveEnvironmentVariables": true
}
}This way, we ensure consistency and keep the api smaller, at the cost of some flexibility for extension authors
@mjbvz That is what I was thinking earlier too, but @isidorn's concern of keeping the configuration service sync while the resolver could be async stopped me. See resolveInteractiveVariables
On second thoughts, do we foresee the use of interactive variables in user/workspace settings?
We could restrict the user/workspace settings to static variables and env variables and we should be able to do what @mjbvz suggests above
Slightly off topic, but are environment variables supported at all yet by vscode? I'm trying to run an msbuild task but I get the error.
"Failed to launch external program msbuild /property:GenerateFullPaths=true.
spawn msbuild ENOENT"
I can't seem to find any solid information about vscode using windows PATH variable. Thanks!
@drwbns Yes VS Code supports env variables. For eg in Debug functionality. May I know where in VS Code are you looking to use it?
@sandy081 I know your question was directed at @drwbns but I wanted to chime in and mention that I would love to use environment variables (and/or custom user-created variables) in settings.json and the workspace's settings.json:
Related: #18709, #17619 (comment)
Also related: pythonVSCode#644
- This would help tremendously when sharing workspace settings among users/platforms.
- This would help a lot when syncing
settings.jsonbetween multiple machines (I have three development machines on different platforms)
Agree with @fredrikaverpil
Currently I'm supporting environment variables and other variables (e.g. $workspaceRoot) in the Python extension using my own (custom) code.
What would be the best way to make Code platform agnostic?
Some variables would need to be added.
Example for some user settings, it would be nice that paths like
${UserSettings}/some folder/some file.json
${UserDocuments}/some folder/some file.json
be resolved to Code user settings path, user documents path and slash to appropriate filepath separator on any platform.
I confirm that currently something like
"emmet.extensionsPath": "${env.APPDATA}\\Code\\User\\Emmet Settings"
will fail.
Yes - please allow code to resolve environment variable references within settings.json. This is especially helpful when syncing settings between multiple instances and paths might be different on different machines.
I support the ${env.VARNAME} syntax across platforms.
It looks like the usage of environment variables in the tasks.json file is now ${env:VARNAME} instead of ${env.VARNAME}. Colon instead of dot. I support the continued usage of the new form in settings.json.
The dot form of environment variables is deprecated (breaking some folks) but should probably remain consistent throughout other config files. See issue 28769.
New form with a colon documented here for tasks.json files and more generally here.
PS. I agree that supporting this would be great and not supporting it is a constant annoyance of hardcoding.
Not a solution, but the temporary workaround I came up with so that I can use the same settings.json on both my work and personal laptops is to use symlinks.
On work laptop I...
sudo ln -s /Users/my-work-username /Users/my-personal-username
And in my settings.json I only use /Users/my-personal-username.
It seems there are two slightly different User Stories here. One is for system Environment Variables to be available for use in settings.json, which comes with the portability concerns. The other is for supporting tasks.json-style Variable Substitution in settings.json, as suggested in #3759. As far as I'm aware this second story doesn't have any cross-platform implications, but because it was closed as a duplicate of this issue it's now unable to be implemented until the more complicated issue is resolved.
I'd suggest that #3759 is re-opened as it seems like it's actually a slightly different issue and could be resolved more easily on its own.
@joaomoreno this issue was in limbo for a while now — now a number of things would be unblocked if this is implemented. Would it make sense to prioritise it up a notch please?
I'd like to +1 this issue as well. We pull our code base into different drive locations (R:, F:, C:\ etc) and so being able to use a variable such as ${folderPath} in our settings.json would be an immense help!
I would like to have full shell power in variables, like we can have in shell variables itself. For example:
I have a project with a lot of different include pathes. Like so: ~/project/platform_name/include where platform_name is one folder from the list of many folders. Current used platform stored in build configuration file. So i can generate path for my editor (vim) by executing something like so
PLATPATH=~/project/$(cat ./build.conf | grep "plat ?=" | sed -E "s/.*= //")/include
echo $PLATPATH
~/project/current_platform/includeand have my path.
I would like to be able to use the same approach in VS Code. Something like this:
"includePath": [
"${workspaceFolder}",
"~/project/${env:$(cat ./build.conf | grep \"plat ?=\" | sed -E \"s/.*= //\")}/include"
],any news on when we'll see this in the main build?
I don't know if this is the same problem, but in my User setting, I can't get environment variables to work. I have a line that looks like this:
"C_Cpp.clang_format_path": "${env:HOME}/.local/bin/clang-format",
and it clang-format doesn't work. I've also tried ${env.HOME} and ${HOME}. If I replace the variable with /Users/benlindsay then clang-format works fine. Does this stuff about settings.json also apply to settings.json in User settings as opposed to just individual projects' settings.json?
@benlindsay Resolving env variables is not supported by default in settings.json. Corresponding extension has to resolve it.
there's an extension that does that?
@chabad360 I'm guessing @sandy081 means that it's up to whatever extension a particular setting belongs to is responsible for handling variable expansion, so in my case the C/C++ extension
but from what I'm understanding, settings.json as a whole doesn't support env variables period. so how does it make a difference which extension it is?
my guess is that in my case the string "${env:HOME}/.local/bin/clang-format" gets passed to the C/C++ extension for the clang_format_path variable, and then the extension can do whatever it wants with that string, and I'm hoping that the extension will read my environment variables and correctly expand it.
there's an extension that does that?
Yes indeed, the Python extension
has been doing this from day one.
so your saying that its just an issue with VS not set to process the env vars.
Now my question is how come that hasn't made it to master already?
Please read the above comment and my respinse. It's upto extensions to do this, not VSCode. So not sure why you are asking about code not making into master.
issue with VS not set to process the
It is by design, hence it's not an issue.
It seems that for the sake of consistency across all the settings exposed by extensions and VS Code itself, the substitution of environment variables should be handled by the framework.
There are so many extensions that don't support environment variable replacement currently and that is a big hindrance when working on multiple platforms or machines.
It would be much easier for the editor to do this replacement than to get thousands of extensions to do it in a consistent way.
Just my 2 cents.
There are a number of people requesting this to be a feature in vscode. This is an open source project guys, please feel free to submit a PR.
I would, but i dont know typescript all that much, and even if I would I'm not sure where to start (I.e. where is the settings.json parser stored?)
Thanks, will have a look at it.
@DonJayamanne can you point me to where in the Python add-on, the file responsible for parsing env vars is? (as I said before typescript is not my thing, so I need a good reference to work from)
Edit: Nevermind, found it myself. but can someone point me to where vscode handles them for debugging?
@DonJayamanne why doesn't VS code simply support .js anywhere .json is? It would allow for an extreme custom and dynamic behavior.
allow for an extreme custom and dynamic behavior.
And dangerous too.
.js is supported via extensions.
And dangerous too.
@DonJayamanne I don't fully understand the concern, but at the same time, I don't have full domain knowledge. Being a scriptable editor danger/security is def not a priority.
.jsis supported via extensions.
This is great news! Mind pointing me in the right direction? 👍
In my instance I am trying to have dynamic launch.js that scans all apps in a repo and adds DEBUG/RUN configurations from a template.
From what I've looked at, the reason why this isn't implemented is because of how the settings.json parser is written: It simply determines what the specific setting belongs to and what its value is, and then passes that on straight to the system responsible for it. It's not actually dealing with any of the values.
In my instance I am trying to have dynamic launch.js that scans all apps in a repo and adds DEBUG/RUN configurations from a template.
This can be achieved today using the DebugConfigurationProvider class
An explicit "early evaluation" mechanism would reconcile all needs:
e.g. by surrounding variable with <>
"param": "${<env:MYVAR>}"
In this case, the param setting would be evaluated immediately during parsing.
In any case, we need some kind of solution with all due haste.
It is critical for cross-platform projects which often have incompatible paths, etc.
@andreyorst This is completely unnecessary: just save the results of that command in an environment variable, and use that variable in the settings.
I know this issue is very old, but I do not understand the need to interface with the host system.
Here is a simple proposal, FWIW (happy to hear criticisms):
Environment files
These would be stored as "flat" JSON objects (i.e. no nesting), in a file environment.json.
Constraints could be applied to the variable names (e.g. no space or brackets), and values (e.g. string or numeric only); any failed validation would result in the variable being ignored (maybe with a warning).
This file could be created both globally in the user's VSCode folder (~/.vscode on Linux/OSX), or locally in the workspace config folder (./.vscode). The latter would override any variable defined globally, when resolving the settings (both global and local).
Advantages
Using environment files would prevent having to interface with the host system, which seems quite difficult when considering the cross-platform constraint. Instead, it is easy for us (users) to generate the environment file with a script, say at login-time. For example with Python:
import os, json
with open( os.path.expanduser('~/.vscode/environment.json'), 'w' ) as envfile:
json.dump( dict(iter(os.environ.items())), envfile, indent=2 )It would also allow users to include additional variables in their VSCode environment, without polluting the system's environment. I would expect VSCode to issue an error if some variables in my settings cannot be resolved (but it would be better to ignore only those settings that failed, rather than all of them).
From the perspective of collaborative development, this would also allow sharing workspace settings directly into a repository; each developer would then fill in the specifics in their own environment (e.g. the path of "such" executable on the local machine), without needing to edit the settings themselves.
Syntax
Not a deal-breaker for me, but I think either $(variable) or ${variable} would be clearer than %VARIABLE%.
However, there's one problem.
The debugger can now reference settings defined in settings.json (via the following PR using the syntax ${config.python.pythonPath}).
Unfortunately the variable${workspaceRoot}doesn't get resolved in the debugger. I cannot implement a work around either as the workspace Root path isn't available in the debugger either.
I know I'm late to the game with this, but I'm seeing this same issue occurring in tasks. E.g. - I want to create a task to perform my pip install -r requirements.txt inside my virtualenv, using the below definition results in this error message > Executing task: ${workspaceFolder}/venv/bin/python -m pip install -r /Users/rohicks/src/xxx/project/requirements.txt <.
{
"label": "restore",
"type": "process",
"command": "${config:python.pythonPath}",
"args": [
"-m",
"pip",
"install",
"-r",
"${workspaceFolder}/requirements.txt"
]
}Sorry if I missed an existing issue for this problem.
From the perspective of collaborative development, this would also allow sharing workspace settings directly into a repository; each developer would then fill in the specifics in their own environment (e.g. the path of "such" executable on the local machine), without needing to edit the settings themselves.
Syntax
Not a deal-breaker for me, but I think either
$(variable)or${variable}would be clearer than%VARIABLE%.
So if I understand this correctly, all these settings would be stored in a single file? Thus, could be used on a removable hard drive or USB memory stick? For a portable (as in moveable, still platform dependant) work space?
From the perspective of collaborative development, this would also allow sharing workspace settings directly into a repository; each developer would then fill in the specifics in their own environment (e.g. the path of "such" executable on the local machine), without needing to edit the settings themselves.
So if I understand this correctly, all these settings would be stored in a single file? Thus, could be used on a removable hard drive or USB memory stick? For a portable (as in moveable, still platform dependant) work space?
@Protocol-IRK I am not sure what you are trying to get at.
These environment files would be specific to a particular computer; you would have one global file (in ~/.vscode/environment.json), and potentially one local file for each project. Yes you could put these on a USB stick, but I don't see the benefit of doing that: the paths configured inside these files would not be valid on another computer a priori.
The collaborative perspective is more like this; imagine that you are working on a code project with other people, and they are all using VSCode. You would like certain extensions to be configured in a certain way for that project (e.g. compiler version and path, output path for compiled files, etc.), but you cannot include the user preferences for every single developer! Instead, you write workspace preferences using "environment variables" and include that in the code repository. Then it is up to each developper to define these environment variables in their own environment files, which are excluded from the repo.
Is that clearer?
These environment files would be specific to a particular computer; you would have one global file (in
~/.vscode/environment.json), and potentially one local file for each project. Yes you could put these on a USB stick, but I don't see the benefit of doing that: the paths configured inside these files would not be valid on another computer a priori.
Thank you for the detailed reply, and sorry for the slow come back. Hopefully I am not getting confused, but bare with me.
At the moment VSCode uses the Windows global PATH environmental variable to discover where a compiler is, for instance GCC (as far as I am aware - please correct me if I am wrong or have missed something). In the case of a limited user you would possibly not be able to edit this variable. So what I was suggesting is that in the settings.json (or specific language) the additional of custom environmental variables. Being able to declare
Hopefully this is understandable.
Thank you.
Why is there no progress on this issue? The request seems simple enough!
Would be good to have the same possibility in the jsconfig.json. In our runtime environment we have some conflicts standard JS-functions and classes. For example, the implementation of File is different.
We created a typescript definition file to solve this issue, but you have to exclude the default library file (lib.d.ts) and resolve this with the correct include order like this:
{
"compilerOptions": {
"noLib": true // exclude lib.d.ts
},
"include": [
"C:\\Users\\<User>\\AppData\\Local\\Programs\\Microsoft VS Code\\resources\\app\\extensions\\node_modules\\typescript\\lib\\lib.dom.d.ts
]
}Would be good if you can use %AppDataLocal% instead of providing the path to the users app data folder.
Don't know if this belongs to the original issue..?
I have to admit, this seems like this should be a simple issue to fix. What my team is using is a application.code-workspace file that looks similar to this:
"folders": [
{
"name": "React Application",
"path": "."
},
{
"name": "GO Server",
"path": "../goserver"
},
{
"name": "Database Models",
"path": "../models-go"
},
{
"name": "GoLang Utilities",
"path": "../utils-go"
},
{
"path": "../../../.go/src/example.com/example/models-go",
"name": "Models (testing)"
}
],
"settings": {
"git.ignoreLimitWarning": true
}
}
As you can see, this works, but only if everyone is using the exact same file structure for their entire development environment, not just in the workspace. An ideal solution would to be able to use path shortcuts, something akin to this:
{
"path": "~/.go/src/example.com/example/models-go",
"name": "Models (testing)"
}
As a workaround, have a dev script create a directory junction with a fixed path to a user-specific folder, like mklink /H /J C:\AppData %APPDATA%
Then your settings file could look like something like:
{
"typescript.tsdk": "C:/AppData/npm/node_modules/typescript/lib"
}
(Things might be more complicated if multiple devs are sharing a machine; the script would need to delete and recreate the junction.)
Is there a technical reason to keep this on hold for so long (I mean, besides prioritization)? Some arguments stated above:
-
It was decided that some kind of name resolving feature was outside the scope of the configuration module which should always return the "truth", so a new service was created for that. But people are asking to resolve names in a consistent manner across different configuration sections in their settings files. It's not a question about architecture, it's a question about function. That answer sounds to me like "please go complain to the Resolving Service department". It's ok if you decided to have that department, but it's inconsequential for the question raised.
-
The sync vs async argument. I don't think this is relevant, I'd say that nobody here would dare to ask for settings to be interactively entered while vscode is launching. It was suggested to leave interactive variables outside the scope of settings files, which is a very sensible position.
Besides that I'm unable to find another argument against introducing variables into settings files. Now, imagine someone opened a PR adding this functionality, imagine the code were acceptable, would it be rejected because 1 or 2 or another reason I'm honesty having a hard time to figure out, since this FR seems everything but unreasonable. Someone asked for a story: most of my dotfiles are full of ~/whatever paths, this is so common in Unix world that the request for a story for something like ${home}/whatever disconcerts me.
Don't mean to be harsh, just to understand you reasons.
And when I write ~/... is not that I want to spare myself of typing a few characters but to check out my dotfiles from a git repo on another computer under a different user and expect that everything keeps working without having to patch paths.
This issue and #5595 are an almost necessary addition for cross-platform setups. Hope to see them soon!
PS: I would like to contribute but by TypeScript understanding is based on JS knowledge which is only moderately good
Is someone working on this? I would love to have a variable for my users home root like ~/.
Is someone working on this? I would love to have a variable for my users home root like
~/.
I thought this was already working?
@audvareyensid not in my version and I use the latest version (1.36.1)
I work with venvs in python and need to set the python.pythonPath explicitly for every project, would be nice if I could use a ${env:VENV_HOME}
@fenchu you can do that by using .vscode/settings.json inside your project and then just use ${workspaceRoot}/venv/bin/python
For consistency I would opt that the settings service does this. Otherwise every extension will behave slightly different. Since VS Code currently doesn't do this it should be opt in when defining the setting. Something like
"settingXYZ": {
"scope": "resource",
"type": "string",
"resolveVariables": true
}
This needs a new API in VS Code to resolve variables. I do not think we cannot reuse the existing as it is sync and resolving is async. Also it can break the existing extensions.
Why would this for environment variables need to be async?
Because our [configurationResolver]
(https://github.com/microsoft/vscode/blob/master/src/vs/workbench/services/configurationResolver/common/configurationResolver.ts#L12) is async
Because our configurationResolver is async
Would it make sense to make it partly non async. I see why some must be but for example env vars don't.
Probably this needs to separate and exclude those variables which are async.
This is necessary for the Docker extension to use CustomExecution2, which in turn is needed for us to support debugging Blazor apps.
Please support this! It has been 3 yrs...
This should be supported. It allows standardization of variables and workflow across developer environments.
It is unbeliveable that an editor which is famous for being customizable does not allow using environment variables in its configuration.
We have also some issues with this. For two indpendet Plugins we want to use one config variable, e.g. here for the plantuml-server-url:
{
"plantuml.commandArgs": [
"-DGRAPHVIZ_DOT=D:\\Progs\\graphviz\\bin\\dot.exe",
],
"asciidoc.preview.attributes": {
"plantuml-server-url": "http://localhost:8080"
},
"plantuml.render": "PlantUMLServer",
"plantuml.server": "http://localhost:8080"
}
Maybe yaml can be used for such configuration?
For using different color themes on different computers, it would be excellent to be able to use environment variables, perhaps like this:
"workbench.iconTheme": "${env:ICONTHEME}",
"workbench.colorTheme": "${env:COLORTHEME}",This feature is really needed.
I use laptop with ubuntu & IMac.
Whenever I open new java project, I need to set up java.home because ${HOME} is different between ubuntu and macOS.
I want to use java.home at settings.json as follows.
( I use sdkman or jenv to configure java)
"java.home": "${env:HOME}/.sdkman/candidates/java/current"
or
"java.home": "${env:HOME}/.jenv/versions/1.8"
I think this needs to go into configuration API in vscode.d.ts - a new api to resolve configuration value.
This is already totally working for for setting up the integrated shell, for example:
"terminal.integrated.shell.windows": "${env:USERPROFILE}\\.dotnet\\tools\\pwsh.exe",
@kzu / @pbrit Wondering, is this documented anywhere? Or perhaps not yet, because it hasn't been formally closed? Also, just for me to understand, does closing a ticket like this one require first updating the documentation with relevant documentation? (i.e. do we consider a ticket "done" when its relevant features are also documented, or is that not required?)
@kzu Shell might be fine, but sth like this:
"vim.neovimPath": "${env:USERPROFILE}\\scoop\\shims\\nvim.exe",
is still not working.
It would be nice if the env vars work in every property of settings.json