Self-Signed Dev Cert Expired

Question

Self-Signed Dev Cert Expired

quarterhorse opened this issue 3 years ago · 35 comments

quarterhorse commented 3 years ago

Using the current Dockerfile, it looks like the self-signed Dev cert is expired as of 28 March 2020:

Answer 1 · 2021-04-27T12:45:30.000Z

Also, it might be helpful to document where the cert is in the container so it can be installed in the host's store.

Answer 2 · 2021-04-27T15:36:35.000Z

Just a follow up:

@microsoft./generator-sharepoint@1.12.0 has been deprecated and the recommendation is to use 1.11.0
execute gulp trust-dev-cert and cert is located at \node_modules\public-encrypt\test\test_cert.pem rather than at \node_modules@rushstack/debug-certificate-manager/temp/161952882073.pem (which is missing at the end of the gulp command execution)
Install test_cert.pem in the Trusted Root Certification Authority - it'll be issued by localhost [probably don't need to do this]
As mentioned elsewhere, update node_modules/gulp-connect/index.js on line 106 change "return this.server.listen(this.port, this.host, (function(_this){" to "return this.server.listen(this.port, (function(_this){"
execute gulp serve

If your browser is chrome-based (Chrome itself, or the new Edge), you'll need to update your browser settings to allow insecure requests to localhost. For Chrome, go to chrome://flags/#allow-insecure-localhost and update. For Edge, go to edge://flags/#allow-insecure-localhost. Firefox will indicate the request is questionable but in the advanced area allow you to continue.

Using this, I'm able to access both the workbench and host for SharePoint debug package content. I hope this helps someone.

Answer 3 · 2021-04-27T15:37:49.000Z

Oh, in my container, I executed npm i @microsoft/generator-sharepoint@1.11.0 to install the recommended version. I should have been clearer in the first bullet.

In Firefox, if you expand the error message, highlight the URL to the failed js and open it in a new tab, you'll be able to approve access to the file which then will, upon refresh, allow the web part to render. The issue is unblocking access to localhost.

Answer 4 · 2021-04-27T17:33:53.000Z

Thanks for sharing! Since the cert is a part of SPFx, I don't think we can do much about it in the Docker image, right?

Answer 5 · 2021-04-28T11:31:14.000Z

That's a good question and I think the answer might be yes. What I was able to do, after reverting to 1.11.0 per the recommendation (https://www.npmjs.com/package/@microsoft/generator-sharepoint/v/1.12.0) I did run the trust-dev-cert and the cert was valid.

I guess I was just trying to be helpful to others that maybe there's another, perhaps simpler approach that seems to work for me, at getting it running under Docker Desktop compared to what's listed in the known issues on the main page. Also, I was thinking that there might be some folks newer to development that don't realize the limitation with HTTPS and localhost on Chrome-based browsers and it being handled different in Firefox.

Answer 6 · 2021-04-28T13:11:59.000Z

Absolutely. Just wanted to clarify that it's nothing that we can change beyond the guidance you've shared. Thank you!

Answer 7 · 2022-01-19T17:00:01.000Z

If I get it correctly, refreshing certificate needs to be done for every new container. In other words, every time we run docker run ... we need to run gulp trust-dev-cert inside it and we also need to install the new certificate on the client device. In order to not install certificate every time manually, we might want to share snippets for automatic installation. For example, here's a one I'm using for Windows client device:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$store = new-object System.Security.Cryptography.X509Certificates.X509Store(
    [System.Security.Cryptography.X509Certificates.StoreName]::Root,
    "localmachine"
)
$store.open("MaxAllowed");
$store.add($x509Certificate);
$store.close();

Should we reopen this issue?

Answer 8 · 2022-01-19T18:24:57.000Z

Why do you think it's needed to refresh the certificate for each container?

Answer 9 · 2022-01-19T20:43:04.000Z

Here's a test after which I concluded that it's needed every time.

scaffold a new project and install dependencies
create a new container, run gulp serve, check the certificate thumbprint exposed by the web server: A2141F169427B53BA0054EF73AE8E1752E214A93
stop gulp serve, run gulp trust-dev-cert, run gulp serve, check the certificate: A84131B648824741501266215187ED990850EAE0
remove the container, create a new container, run gulp serve, check the certificate: A2141F169427B53BA0054EF73AE8E1752E214A93

so after creating a new container it will be always reset to the old, which is expired

Answer 10 · 2022-01-22T15:53:25.000Z

Thank you for the looking into it. Have you tried installing the certificate just once and when you get an error in the browser after it expired, simply ignore it? I wonder if asking folks to reinstall the certificate on the host each time they start the container isn't too much hassle and if it's really necessary.

Answer 11 · 2022-01-23T19:37:42.000Z

There are two problems with the built in SPFx (gulp-connect, to be more specific) certificate (A2141F169427B53BA0054EF73AE8E1752E214A93, https://npm.runkit.com/gulp-connect/certs/server.crt?t=1642961667180):

it expired in 2020
host name does not match localhost

Firefox allowed to add a security exception
Chrome allowed this also
for Edge I could not find the way to do it:

As I mentioned before, running gulp trust-dev-cert unfortunately only helps until the container is running. However, another solution is to replace these two files:

./node_modules/gulp-connect/certs/server.key
./node_modules/gulp-connect/certs/server.crt

Simply run these lines in the container to update the files:

openssl genrsa 2048 > ./node_modules/gulp-connect/certs/server.key
openssl req -new -x509 -nodes -sha256 -days 365 -key ./node_modules/gulp-connect/certs/server.key -out ./node_modules/gulp-connect/certs/server.crt -subj "/C=US/ST=Uta/L=Lehi/O=Your Company, Inc./OU=IT/CN=localhost" -extensions SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS:localhost"))

After that the container can be terminated and recreated and if this certificate is added to trusted root on the client side, it will be still trusted in a new container.

Let's try to summarise:

For Firefox and Chrome users - just use a standard built in certificate, don't run gulp trust-dev-cert.
For Edge users there seems to be two options

run gulp trust-dev-cert every time when a new container is started and then add the new certificate on the client side (for example, with the script above for Windows)
run the two lines snippet for replacing the certificate in the project files and add the certificate once to the client side

Answer 12 · 2022-01-24T11:09:24.000Z

After that the container can be terminated and recreated and if this certificate is added to trusted root on the client side, it will be still trusted in a new container.

If you're replacing the cert in the node_modules folder in the current project, won't be gone if you use a different project or you remove/reinstall dependencies?

Answer 13 · 2022-01-24T11:44:09.000Z

it will be gone in such cases. So this procedure should be done every time we install dependencies. Similarly to how we need to manipulate with WebpackConfigurationGenerator.js I guess.

Answer 14 · 2022-01-24T14:28:01.000Z

Let me pass this information to SPFx engineering. Meanwhile, for now I think what you suggestes @shurick81 is the only way forward.

Answer 15 · 2022-01-25T13:31:10.000Z

If I try to load an SPFx v1.11.0 project in Edge I'm also getting a warning about the cert:

I can however bypass it just fine:

I'm using Edge Version 97.0.1072.62 (Official build) (x86_64) on macOS

Answer 16 · 2022-01-25T17:01:50.000Z

Ok, perhaps it might be different depending on the corporate policies... I use Version 97.0.1072.69 (Official build) (64-bit)

Answer 17 · 2022-01-26T13:39:02.000Z

Thinking of it some more, I think that's an issue with SPFx itself rather than with using SPFx in Docker specifically. If you'd run an older SPFx project on your host, you'd have the same issue right? Perhaps the best way forward would be to request an update to the SPFx documentation to include a workaround for older version, while ensuring that this issue will no longer happen in the future versions of SPFx.

Answer 18 · 2022-01-26T18:12:52.000Z

Let's try to summarize:

There seems no difference between old and new SPFx: default certificate expired in 2020.
For most cases, the default certificate can be trusted.
gulp trust-dev-cert updates certificate in the system (not per project).
SPFx documentation says to run gulp trust-dev-cert https://docs.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-development-environment#trusting-the-self-signed-developer-certificate. So I guess people who develop SPFx don't really care about default certificate that is expired.
In the future versions of SPFx someone may update default certificate.
running gulp trust-dev-cert makes little sense for users of Docker: A) it should be done every time when the docker starts. B) in Docker the new certificate will not be automatically installed even for Windows users that are used to such behavior.

When you @waldekmastykarz tried to run SPFx v1.11.0 project in Edge, did you use the docker image and you did not run gulp trust-dev-cert or something like this to generate anew cert?

What I could also do is checking if we could include the system certificate update in the Dockerfile so that the system certificate is not expired when user runs a new container.

Answer 19 · 2022-01-27T07:52:00.000Z

There seems no difference between old and new SPFx: default certificate expired in 2020.

I think that's not quite true. If I'm not mistaken, recent versions of SPFx no longer use gulp-connect and instead use a Heft plugin. When browsing to a 1.13.1 project, I can see the browser using a certificate expiring in 2025 whereas older versions show indeed a cert that expires in 2020.

So I guess people who develop SPFx don't really care about default certificate that is expired.

Unless folks have a similar limitation like you do, where they can't navigate around an expired certificate, I think it's a fair assumption that they ignore the warning.

When you @waldekmastykarz tried to run SPFx v1.11.0 project in Edge, did you use the docker image and you did not run gulp trust-dev-cert or something like this to generate anew cert?

Yes, I tested it with Docker. The cert that was used in browser was expired in 2020.

Adding @AJIXuMuK from SPFx engineering for visibility

Answer 20 · 2022-02-01T22:34:59.000Z

Here's the test I am doing,

running in WSL: docker run --rm -it -v $(pwd):/usr/app/spfx -p 4321:4321 -p 35729:35729 waldekm/spfx:1.13.1
running inside the container:

yo @microsoft/sharepoint --solution-name helloworld --component-type webpart --component-name hello-world-webpart --component-description "HelloWorld web part" --is-domain-isolated --framework none --environment spo --skip-feature-deployment false
cd helloworld/
gulp serve

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Saturday, March 28, 2020 4:26:16 PM
A2141F169427B53BA0054EF73AE8E1752E214A93

Answer 21 · 2022-02-03T15:26:50.000Z

Thank you for the additional information. Does the output change when you run gulp trust-dev-cert in the container?

Answer 22 · 2022-02-03T15:44:20.000Z

You are the most welcome! Now I run it like this:

running in WSL: docker run --rm -it -v $(pwd):/usr/app/spfx -p 4321:4321 -p 35729:35729 waldekm/spfx:1.13.1
running inside the container:

yo @microsoft/sharepoint --solution-name helloworld --component-type webpart --component-name hello-world-webpart --component-description "HelloWorld web part" --is-domain-isolated --framework none --environment spo --skip-feature-deployment false
cd helloworld/
gulp serve --nobrowser

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Saturday, March 28, 2020 4:26:16 PM
A2141F169427B53BA0054EF73AE8E1752E214A93

In the container that is still running, I put Ctrl + C to stop ongoing gulp serve and then I shoot this:

gulp trust-dev-cert
gulp serve --nobrowser

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Monday, February 3, 2025 4:40:45 PM
829A0F0ACB6B1EFB41122DFA2842B31E66B35E20

Answer 23 · 2022-02-09T12:57:24.000Z

One last question: when you restart the container, does it use the old (default) or the new certificate?

Answer 24 · 2022-02-09T17:44:21.000Z

Do you mean these steps?

running in WSL: docker run --rm -it -v $(pwd):/usr/app/spfx -p 4321:4321 -p 35729:35729 m365pnp/spfx:1.13.1
running inside the container:

yo @microsoft/sharepoint --solution-name helloworld --component-type webpart --component-name hello-world-webpart --component-description "HelloWorld web part" --is-domain-isolated false --framework none --environment spo --skip-feature-deployment false
cd helloworld/
gulp serve --nobrowser

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Saturday, March 28, 2020 4:26:16 PM
A2141F169427B53BA0054EF73AE8E1752E214A93

In the container that is still running, I put Ctrl + C to stop ongoing gulp serve and then I shoot this:

gulp trust-dev-cert
gulp serve --nobrowser

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Sunday, February 9, 2025 6:40:57 PM
89748BB3BED8126528D5B5363B838AC6FE9FE2B2

Stop/delete the container, run again from the same directory in WSL: docker run --rm -it -v $(pwd):/usr/app/spfx -p 4321:4321 -p 35729:35729 m365pnp/spfx:1.13.1
running inside the container:

cd helloworld/
gulp serve --nobrowser

On the same machine, running the PowerShell Snippet:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs:

Saturday, March 28, 2020 4:26:16 PM
A2141F169427B53BA0054EF73AE8E1752E214A93

Answer 25 · 2022-02-10T12:24:50.000Z

Yes, exactly these steps. So the conclusion is that it reverts to the original cert. Do you know where the certificate is stored? I assume not in node_modules because that's mapped to a folder on your host which should be persisted across executions.

Answer 26 · 2022-02-11T10:49:47.000Z

I took some time of digging :)

The sequence is like this:

/node_modules/@microsoft/gulp-core-build-serve/lib/ServeTask.js is executing _loadHttpsServerOptionsAsync before running gulpConnect.server:

const httpsServerOptions = await this._loadHttpsServerOptionsAsync();

...

gulpConnect.server({
    https: httpsServerOptions,
    livereload: true,
    // eslint-disable-next-line @typescript-eslint/ban-types
    middleware: () => [this._logRequestsMiddleware, this._enableCorsMiddleware],
    port: port,
    root: path.join(rootPath, this.taskConfig.rootFolder || ''),
    preferHttp1: true,
    host: hostname
});

_loadHttpsServerOptionsAsync is calling /node_modules/@rushstack/debug-certificate-manager/lib/CertificateManager.js, in its turn it's calling CertificateStore, which is checking /home/spfx/.rushstack/rushstack-serve.key. NOTE, this is a path in the container image, not the mounted volume. If the file is not found, CertificateStore returns undefined.
gulpConnect is checking if certificate is a part of https parameters. If it is not, it is using this default certificate: ./node_modules/gulp-connect/certs/server.key. NOTE that this time it is looking in the mounted volume, not in the container image

Answer 27 · 2022-02-11T11:25:27.000Z

And when you run gulp trust-dev-cert I suppose it refreshes the cert in the .rushstack folder which isn't persisted, correct?

Answer 28 · 2022-02-11T12:11:14.000Z

yes, that's right

Answer 29 · 2022-02-12T00:43:18.000Z

maybe we should just put some lines in our Dockerfile for updating /home/spfx/.rushstack/rushstack-serve.key?

something like this:

openssl genrsa 2048 > /home/spfx/.rushstack/rushstack-serve.key
openssl req -new -x509 -nodes -sha256 -days 365 -key /home/spfx/.rushstack/rushstack-serve.key -out /home/spfx/.rushstack/rushstack-serve.pem -subj "/C=US/ST=Uta/L=Lehi/O=Your Company, Inc./OU=IT/CN=localhost" -extensions SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS:localhost"))

Hugo is also solving it in https://github.com/pnp/sp-dev-fx-webparts/blob/main/samples/react-enhanced-powerapps/.devcontainer/spfx-startup.sh but we might try a simpler solution that is integrated directly in the image?

Answer 30 · 2022-02-14T08:28:28.000Z

Wouldn't it be easier to call the gulp trust-dev-cert task that does the same?

Answer 31 · 2022-02-14T08:40:51.000Z

well, to run this task you will need to have an spfx solution scaffolded?

Answer 32 · 2022-02-14T15:30:10.000Z

Will you have the /home/spfx/.rushstack/rushstack-serve.key file without scaffolding a project?

Answer 33 · 2022-02-14T19:23:55.000Z

Maybe I am explaining it ambiguously, but here's what I tested now.

I built a local image with this file:

FROM node:14.18.0

EXPOSE 4321 35729

ENV NPM_CONFIG_PREFIX=/usr/app/.npm-global \
  PATH=$PATH:/usr/app/.npm-global/bin

VOLUME /usr/app/spfx
WORKDIR /usr/app/spfx
RUN useradd --create-home --shell /bin/bash spfx && \
    usermod -aG sudo spfx && \
    chown -R spfx:spfx /usr/app

USER spfx

RUN npm i -g gulp@4 yo @microsoft/generator-sharepoint@1.13.1
RUN bash -c "mkdir /home/spfx/.rushstack && openssl genrsa 2048 > /home/spfx/.rushstack/rushstack-serve.key && openssl req -new -x509 -nodes -sha256 -days 365 -key /home/spfx/.rushstack/rushstack-serve.key -out /home/spfx/.rushstack/rushstack-serve.pem -subj '/C=US/ST=Uta/L=Lehi/O=NpM, Inc./OU=IT/CN=localhost' -extensions SAN -config <(cat /etc/ssl/openssl.cnf <(printf '[SAN]\nsubjectAltName=DNS:localhost'))"

CMD /bin/bash

run docker run --rm -it -v $(pwd):/usr/app/spfx -p 4321:4321 -p 35729:35729 spfx-build:0
in the container run

yo @microsoft/sharepoint --solution-name helloworld --component-type webpart --component-name hello-world-webpart --component-description "HelloWorld web part" --is-domain-isolated false --framework none --environment spo --skip-feature-deployment false
cd helloworld/
gulp serve --nobrowser

In PS on the same machine where Docker is running, I run this:

$tcpClient = New-Object -TypeName System.Net.Sockets.TcpClient;
$tcpClient.Connect("localhost", 4321);
$tcpStream = $tcpClient.GetStream();
$callback = { param($sender, $cert, $chain, $errors) return $true };
$sslStream = New-Object -TypeName System.Net.Security.SslStream -ArgumentList @($tcpStream, $true, $callback);
$sslStream.AuthenticateAsClient('');
$certificate = $SslStream.RemoteCertificate;
$x509Certificate = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate2 -ArgumentList $certificate
$x509Certificate.NotAfter
$x509Certificate.Thumbprint

This outputs

Tuesday, February 14, 2023 7:58:26 PM
5ECB9062B0E1EC90DD597980501435E612FF6287

So the concept is that we create a normal certificate when we create an image. And for the end user it works right from the beginning, without need to run additional certificate creation commands. If user adds it to the browser, it will work until user starts using some other certificate. Alternatively, we can generate a certificate, put it into https://github.com/pnp/docker-spfx repo and put it in the image in all versions so that users don't have to add new cert to browsers every time they use new image version.

Answer 34 · 2022-02-15T07:45:28.000Z

Thanks for the additional information. I think I missed mkdir /home/spfx/.rushstack which you use to create the folder. The reason why I'd prefer to use gulp trust-dev-cert instead of us creating the cert manually using openssl, is because it isolates us from updates in the underlying infrastructure. Should the SPFx team decide to change how they manage a certificate, we'd need to update our guidance, whereas if we suggest using gulp trust-dev-cert it'll be one approach that works across all versions of the image.
Because you won't need certificate until you serve the project, by which time you already have all dependencies needed to run gulp trust-dev-cert, I don't think there's anything preventing us from using it, right?

Answer 35 · 2022-02-15T08:47:56.000Z

Yes, I agree, that is not any showstopper but just a minor inconvenience. I think we discovered a lot of possible solutions here, which might be helpful for future users, even if we don't implement them into image. So let's focus on more pressing issues :)