hydraulic-software/conveyor

Self signed Windows download link fails

Closed this issue · 13 comments

The link on the generated page for self-signed windows apps generates a .Content link that fails.

Website:

image

Executing the command:

image

Changing it to .RawContent produces lots of warnings, but it executes and installs correctly:

image

Full output:

HTTP/1.1 : The term 'HTTP/1.1' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:1 char:1
+ HTTP/1.1 200 OK
+ ~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (HTTP/1.1:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Connection: : The term 'Connection:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a
path was included, verify that the path is correct and try again.
At line:2 char:1
+ Connection: keep-alive
+ ~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Connection::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Content-MD5: : The term 'Content-MD5:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a
path was included, verify that the path is correct and try again.
At line:3 char:1
+ Content-MD5: BGXZYgNEuB9jsnyse4HvjA==
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Content-MD5::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-request-id: : The term 'x-ms-request-id:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,
or if a path was included, verify that the path is correct and try again.
At line:4 char:1
+ x-ms-request-id: 0595dae8-f01e-0030-7d3c-a79caa000000
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-request-id::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-version: : The term 'x-ms-version:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:5 char:1
+ x-ms-version: 2020-04-08
+ ~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-version::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-creation-time: : The term 'x-ms-creation-time:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try again.
At line:6 char:1
+ x-ms-creation-time: Wed, 03 Aug 2022 12:51:08 GMT
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-creation-time::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-lease-status: : The term 'x-ms-lease-status:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try again.
At line:7 char:1
+ x-ms-lease-status: unlocked
+ ~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-lease-status::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-lease-state: : The term 'x-ms-lease-state:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try again.
At line:8 char:1
+ x-ms-lease-state: available
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-lease-state::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-blob-type: : The term 'x-ms-blob-type:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,
or if a path was included, verify that the path is correct and try again.
At line:9 char:1
+ x-ms-blob-type: BlockBlob
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-blob-type::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Content-Disposition: : The term 'Content-Disposition:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of
the name, or if a path was included, verify that the path is correct and try again.
At line:10 char:1
+ Content-Disposition: attachment; filename=launch.win.txt
+ ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Content-Disposition::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

filename=launch.win.txt : The term 'filename=launch.win.txt' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:10 char:34
+ Content-Disposition: attachment; filename=launch.win.txt
+                                  ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (filename=launch.win.txt:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

x-ms-server-encrypted: : The term 'x-ms-server-encrypted:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling
of the name, or if a path was included, verify that the path is correct and try again.
At line:11 char:1
+ x-ms-server-encrypted: true
+ ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (x-ms-server-encrypted::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Fastly-Restarts: : The term 'Fastly-Restarts:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,
or if a path was included, verify that the path is correct and try again.
At line:12 char:1
+ Fastly-Restarts: 1
+ ~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Fastly-Restarts::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Age: : The term 'Age:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:13 char:1
+ Age: 122
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (Age::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

X-Served-By: : The term 'X-Served-By:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a
path was included, verify that the path is correct and try again.
At line:14 char:1
+ X-Served-By: cache-vie6358-VIE
+ ~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (X-Served-By::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

X-Cache: : The term 'X-Cache:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:15 char:1
+ X-Cache: HIT
+ ~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (X-Cache::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

X-Cache-Hits: : The term 'X-Cache-Hits:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:16 char:1
+ X-Cache-Hits: 1
+ ~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (X-Cache-Hits::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

X-Timer: : The term 'X-Timer:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path
was included, verify that the path is correct and try again.
At line:17 char:1
+ X-Timer: S1659533097.106300,VS0,VE111
+ ~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (X-Timer::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Accept-Ranges: : The term 'Accept-Ranges:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or
if a path was included, verify that the path is correct and try again.
At line:18 char:1
+ Accept-Ranges: bytes
+ ~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Accept-Ranges::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Content-Length: : The term 'Content-Length:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name,
or if a path was included, verify that the path is correct and try again.
At line:19 char:1
+ Content-Length: 1779
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Content-Length::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Content-Type: : The term 'Content-Type:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:20 char:1
+ Content-Type: application/octet-stream
+ ~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Content-Type::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Date: : The term 'Date:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:21 char:1
+ Date: Wed, 03 Aug 2022 13:24:57 GMT
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Date::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

ETag: : The term 'ETag:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:22 char:1
+ ETag: "0x8DA754ED650B762"
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (ETag::String) [], CommandNotFoundException                                                                                  + FullyQualifiedErrorId : CommandNotFoundException                                                                                                                                                                                                                                                                                        Last-Modified: : The term 'Last-Modified:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or   if a path was included, verify that the path is correct and try again.
At line:23 char:1
+ Last-Modified: Wed, 03 Aug 2022 12:51:08 GMT
+ ~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Last-Modified::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Server: : The term 'Server:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:24 char:1
+ Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
+ ~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Server::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Via: : The term 'Via:' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:25 char:1
+ Via: 1.1 varnish
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (Via::String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Thanks for the bug report.

I think the issue here is that GitHub is serving the .txt file as application/octet-stream which isn't correct. Windows then hands us back a byte array instead of a string as expected. From reading around it seems that maybe there's no way to fix this via the GUI, but, you can specify a custom MIME type when uploading assets via the GitHub API. The .txt extension is an attempt to persuade web servers to do the right thing here, but apparently, GitHub's CDN is not such a web server.

A workaround is to just put the launch.win.txt next to your download.html file on your own web server (ditto for the Mac script), and then edit it post-generate to replace the URL with the one that isn't on GitHub releases. Your web server should serve it as text/plain as PowerShell will be happy.

In the product there are three possible fixes here:

  1. Very quick: document that when using the combination of self-signing and GitHub releases, you need to work around the MIME type issue by hosting the launch scripts elsewhere and editing the download HTML.
  2. Reasonably quick: allow the generated HTML/text assets to be given a different URL to the download assets. Then you can stick the generated HTML and launch scripts in GitHub Sites or your own company web server, whilst the actual binaries and update metadata is still hosted on GitHub Releases.
  3. Best but a bit less quick: teach Conveyor to upload files to GitHub Releases via the API automatically, setting the correct MIME types as it goes.

Support for making GitHub releases is already tracked internally in CO-86. It won't make the next release. The docs can be fixed now, however.

fwiw, specfically setting the content type does not seem to work either

iex (iwr -ContentType "text/plain" "https://github.com/HebiRobotics/Scope/releases/latest/download/launch.win.txt").Content

Would downloads from compliant servers break when invoking .RawContent ?

Well, RawContent will try and interpret the HTTP headers as PowerShell expressions as you saw above, so that seems a bit risky.

-ContentType seems to set the MIME type for uploads, not downloads.

This does seem to work though:

iex (iwr "https://github.com/HebiRobotics/Scope/releases/latest/download/launch.win.txt").ToString()

We'll update the generated HTML to use that command instead.

Well, RawContent will try and interpret the HTTP headers as PowerShell expressions as you saw above, so that seems a bit risky.

wow, I glanced over the warnings and didn't realize that it includes headers 👍

This does seem to work though

That looks like a good option! If that works independent of the MIME type, maybe you can also get rid of the workaround .txt extension?

Could do, though it seems safer to double up the protection. I guess you'd prefer it to end in a .ps1 extension?

Chocolatey and the other services where I've seen this before seem to to call it install.ps1. It looks more professional than the .txt and makes it clear that it's part of the files that need to be uploaded.

Fwiw, Chocolatey uses iex ((New-Object System.Net.WebClient).DownloadString('<url>'), which does work as well. I don't know whether that is more robust, but the extra verbosity makes it immediately obvious that it's downloading a file and executing it.

That expression does look safer, yeah. Relying on a ToString() method is not really ideal. I'll switch us to using what Chocolatey does and then we can rename the script file too.

By the way, do you plan to use the self-signing mode for real / in production, or are you using it now just for testing and dev purposes?

No, I just haven't gotten to the signing point yet.

I'm also deliberately testing some things so I can provide some feedback to you. Our app is fairly complex and should be good for testing various corner cases.

On a blank VM this fails due to scripts being disabled in the execution policies. Calling the full Chocolatey command makes it work

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://github.com/HebiRobotics/Scope/releases/latest/download/launch.win.ps1'))

OK, that's what we'll go with then.

@ogpublic Ovi, can you take this one? It's CO-207.

Sure thing.