The "any" definition in the schema does not allow for `number`
sirosen opened this issue · 10 comments
The parameters in a template pipeline have a field, default, defined as "any".
"any" can be seen here:
azure-pipelines-vscode/service-schema.json
Lines 30 to 46 in 2e9781b
It does not include numeric types. The following pipeline therefore fails validation:
parameters:
- name: vals
default: [1, 2]
jobs:
- ${{ each val in parameter.vals }}:
- job: foo
steps:
- bash: echo ${{ val }}My understanding is that the above is valid, and that these values will be coerced to strings appropriately.
This is once again a problem with the ADO-defined schema, and unfortunately out of the control of this extension :(.
From #411 (comment) it looks like this behavior might be intentional, which would be very confusing...I need to sit down sometime and figure this all out, then decide who to file bugs on.
Hi @sirosen, this issue isn't retated to this extension. You can submit the same issue to Microsoft developer community. Do you have any additional questions regarding azure-pipelines-vscode extension itself?
You can submit the same issue to Microsoft developer community.
No. This is where I draw the line as a developer of FOSS software who is only working with this schema on personal time. The schema is in this repo.
Judging from @50Wliu's response, the schema is actually provided by some other upstream team.
I should note that the schema is listed in SchemaStore as being available here. That's a community project, but if there's a more accurate location that I and others should be going to get the schema, tell me and I'll put in a PR to update SchemaStore.
@sirosen, the schema in this repository is generated on the Azure DevOps server, the VSCode extension only is just a schema consumer, so to fix this schema - first need to fix the schema on Azure DevOps side. The schema in Azure DevOps is supported by another team, and to fix this problem it would be better to create a ticket on the Microsoft developer community, once the schema will be updated we will be able to update it on the extension side.
@golubev-ivan is right. As for why it appears like we own the schema (and why I assume Schema Store points to this repo as the source location) - the API endpoint to retrieve the schema is organization-specific because organizations can install extensions that add custom tasks. This extension doesn't yet support retrieving the org-specific schemas on-demand, so it provides a baseline by using a dummy ADO organization to copy over the base schema on every extension release (see https://github.com/microsoft/azure-pipelines-vscode/blob/main/RELEASE.md#bumping-service-schema).
The real source of truth is dependent on your ADO organization, and available using https://docs.microsoft.com/en-us/rest/api/azure/devops/distributedtask/yamlschema/get.
it provides a baseline by using a dummy ADO organization to copy over the base schema on every extension release
Is that location publicly available?
Better yet, can you ask the Azure Pipelines developers provide the base schema from a URL where we could GET it and list it in SchemaStore, to cut out the middleman? Not only would this drop the need for the VSCode extension maintainers to keep a dummy org around (unless it's needed for other things too), it would also clarify who owns the schema and where to go to report bugs with it.
I hope you can understand my distaste for mucking around in a support forum when I'm trying to report a bug.
I'm not trying to be belligerent about this, but there's a history of discussion in this repo about issues pertaining to the schema and schema validation. Enough so that there's an accurate-yaml-validation label on this bug report.
For example, looking at past reported issues in that forum, it's easy to find things like this bug report about missing support for template expressions which just refers back to #187 on this repo! I shouldn't think it's at all surprising that the support personnel manning that forum would prefer to push tickets to a more technical space for discussion.
Plus, there are tickets like #411, where there wasn't any runaround about "this isn't the right place for this".
I'll bend in this case, and put in a message in that forum. But I think it should be clear why the decision to boot these tickets out of this repo appears arbitrary.
Is that location publicly available?
Nope (most likely for compliance reasons even if there's nothing stored in it).
Better yet, can you ask the Azure Pipelines developers provide the base schema from a URL where we could GET it and list it in SchemaStore, to cut out the middleman?
This would be great! I'll defer to @golubev-ivan on this.
I hope you can understand my distaste for mucking around in a support forum when I'm trying to report a bug.
I understand, it's not very clear that this repository doesn't own the schema. That's something we can do better at clarifying. What makes it even trickier is that parts of the schema (builtin tasks at https://github.com/microsoft/azure-pipelines-tasks) are controllable.
This is getting into the weeds, but "accurate-yaml-validation" more-or-less covers three areas:
- Validation issues with the "core" schema (non-tasks) are out of our control, but the standard has sort of been to keep them open to reduce the number of duplicate issues we get. Moving forward I think the best course of action is still to keep them open, but also to file Developer Community tickets.
- Validation issues with tasks are also kept open, and we try to be good at upstreaming PRs to azure-pipelines-tasks to fix those, then closing the issue once they're ingested into the schema endpoint.
- Validating correctness beyond the schema, such as template expressions, unique stage names, or only using runtime variables when allowed. And technically those issues belong in azure-pipelines-language-server, but again we're treating this repository as the landing pad because they're so tightly coupled.
Essentially, it boils down to "this repository is an aggregator of validation-related issues for visibility. We can answer questions, but if you're looking for something to be fixed, that ultimately needs to happen elsewhere", elsewhere being Developer Community/azure-pipelines-tasks/azure-pipelines-language-server.
I can't provide the base schema and as far as I know, there is no way to get the base schema that isn't related to the organization now. Right now you can only get a scheme related to the organization, as @50Wliu mentioned. Adding the ability to get the base schema through the API requires changes on the API side.
The real source of truth is dependent on your ADO organization, and available using https://docs.microsoft.com/en-us/rest/api/azure/devops/distributedtask/yamlschema/get.
Predictably, the response from filing this bug report in that forum is less than satisfactory. The same holds for the other bug I took the time to report there.
All I wanted out of this was to somehow get a message to the schema maintainers that their schema has inaccuracies which make it very difficult to use for validation outside of the language-server. If the org decides that's not a problem because it works for the language server, and you want to close this as a wontfix, then please just do that.
This reply made me think that "okay, this will get upstreamed appropriately." But at this point, it's starting to seem that there's no way for me -- a downstream software developer trying to work with this schema -- to report issues directly to the team responsible. I want to report problems with the schema in a place where I'm not going to have to explain myself eight times over, and the repo maintainers don't want to hear about it because the schema is officially not their area of concern. It seems that we're just talking in circles.
I'm very frustrated at this point, but not insensitive to the repo maintainers' concerns. The fact that the schema is publicly visible here is pretty much an accident of software development, not any kind of carefully considered decision.
However, here's where I'd point out an (at least apparent) inconsistency:
Essentially, it boils down to "this repository is an aggregator of validation-related issues for visibility. We can answer questions, but if you're looking for something to be fixed, that ultimately needs to happen elsewhere", elsewhere being Developer Community/azure-pipelines-tasks/azure-pipelines-language-server.
If this is the place where validation issues go, then this is where validation issues go.
There's no more appropriate place for me to report an issue with the schema. The responses on the "Developer Community forum" demonstrate that it is a much less appropriate place to try to describe these problems. The personnel there don't even understand the bugs being reported.
I understand and respect that the team which maintains this repo cannot be expected to fix these issues. But you could just accept the bug reports, rather than asking me to waste time in a "support forum".
Hello @sirosen, I'm closing this ticket, since we can't help you with issues that are related to the schema, our team doesn't support Azure DevOps YAML schema. Extension in fact is the same consumer of ADO schema as you and all the questions related to schema outside our ownership.