WASdev/ci.docker

parseAlgParameters failed: PBES2 AlgorithmParameters not available on FIPS

Closed this issue · 16 comments

Hello,

On an OCP 4.12 FIPS cluster, using ibmcom/websphere-liberty:22.0.0.13-kernel-java11-openj9-ubi

Following https://www.ibm.com/support/pages/fips-certified-cryptography-ibm-semeru-runtimes
So, using the JVM options :

-Dsemeru.fips=true
-Djava.security.debug=semerufips
-Djavax.net.ssl.trustStore=NONE
-Djavax.net.ssl.trustStoreProvider=SunPKCS11-NSS-FIPS
-Djavax.net.ssl.keyStoreType=PKCS11
-Djavax.net.ssl.keyStore=NONE
-Djavax.net.ssl.keyStoreProvider=SunPKCS11-NSS-FIPS

The following error is generated :

[2/15/23, 10:55:24:652 CET] 00000027 WSKeyStore W CWPKI0809W: There is a failure loading the defaultKeyStore keystore. If an SSL configuration references the defaultKeyStore keystore, then the SSL configuration will fail to initialize.
[2/15/23, 10:55:24:652 CET] 00000027 WSKeyStore E CWPKI0033E: The keystore located at /opt/ibm/wlp/output/defaultServer/resources/security/key.p12 did not load because of the following error: parseAlgParameters failed: PBES2 AlgorithmParameters not available

I found https://access.redhat.com/solutions/6954451 which is talking about a keytool issue in FIPS mode :
FIPS mode does not yet support password-based encryption.
If you want to use password-based encryption, you must disable FIPS mode when running keytool.
For example, you could disable FIPS mode for just the keytool process with the -Dcom.redhat.fips=false system property:

Assuming perhaps liberty startup is using keytool, I tried -Dcom.redhat.fips=false JVM option.
But, it didn't solve.
Please, can you advise ?

keytool is indeed used. @arturdzm could you please take a look when you get a chance? thanks

Probably this script should be changed in case of FIPS :

keytool -import -storetype pkcs12 -noprompt -keystore "${TRUSTSTORE_FILE}" -file "${CERT_FOLDER}/${CA_FILE}" \

@mmouly The Liberty documentation for FIPS mode with Semeru Runtime is at https://openliberty.io/docs/latest/enable-fips.html. Have you completed steps 3-5?

FYI, it's same as the Example 3: Configuring Open Liberty to use FIPS mode section in Semeru documentation: https://www.ibm.com/support/pages/fips-certified-cryptography-ibm-semeru-runtimes

Hello @leochr ,
yes we manage the liberty FIPS configuration using this documentation.
However, I think there is no direct relation with the issue concerning the /opt/ibm/wlp/output/defaultServer/resources/security/key.p12 generation.

According to https://www.ibm.com/docs/en/was-liberty/base?topic=liberty-ssl-defaults-in
key.p12 is always automatically generated.
But, this file is never referenced in our FIPS configuration as our keystore usage is this way :

<keyStore id="defaultKeyStore" password="Liberty" location=“/tmp/pkcs11cfg.cfg” type=“PKCS11” fileBased=“false” provider=“SunPKCS11-NSS-FIPS” />

Logs are :

[2/24/23, 9:10:30:574 CET] 00000027 WSKeyStore E CWPKI0033E: The keystore located at /opt/ibm/wlp/output/defaultServer/resources/security/key.p12 did not load because of the following error: parseAlgParameters failed: PBES2 AlgorithmParameters not available [2/24/23, 9:10:30:576 CET] 00000027 WSKeyStore W CWPKI0809W: There is a failure loading the defaultKeyStore keystore. If an SSL configuration references the defaultKeyStore keystore, then the SSL configuration will fail to initialize. [2/24/23, 9:10:30:603 CET] 00000027 SystemErr R javax.net.ssl|DEBUG|27|Config-Thread-11|2023-02-24 09:10:30.590 CET|SSLContextImpl.java:428|System property jdk.tls.client.cipherSuites is set to 'null' [2/24/23, 9:10:30:604 CET] 00000027 SystemErr R javax.net.ssl|DEBUG|27|Config-Thread-11|2023-02-24 09:10:30.604 CET|SSLContextImpl.java:428|System property jdk.tls.server.cipherSuites is set to 'null' [2/24/23, 9:10:30:617 CET] 00000027 SystemErr R javax.net.ssl|DEBUG|27|Config-Thread-11|2023-02-24 09:10:30.617 CET|SSLCipher.java:464|jdk.tls.keyLimits: entry = AES/GCM/NoPadding KeyUpdate 2^37. AES/GCM/NOPADDING:KEYUPDATE = 137438953472

We have a FIPS cluster available. So, if you want to get more detail or have a direct look at it, we can provide you the connection information in DM ?
Thanks for your help.

@leochr any news ? Do we needs to create a case for this issue ?

leochr commented

Are you mounting your certificates at /etc/x509/certs? It seems like that might be interfering with Liberty image's start-up routine to import certificates to default keystore

mmouly commented

Hello @leochr ,
We don't configure this. Anyway, I had a check and this directory doesn't exist

sh-4.4$ ls -la /etc/x509 ls: cannot access '/etc/x509': No such file or directory
As we said, we can share the access to the FIPS cluster. So, you can have material to investigate.

leochr commented

@mmouly Okay. Thanks for checking and confirming. Yes, please send the cluster details directly to me along with your Dockerfile.

mmouly commented

Thanks @leochr ,

Concerning Dockefile, here is the link :
https://github.com/DecisionsDev/odm-ondocker/blob/master/decisioncenter/Dockerfile

Many Thanks for your help

leochr commented

@mmouly Thanks for providing access to the cluster. I had a look into the decisioncenter pod. Below I've included my analysis, next steps and ways to avoid the issue.

Liberty image includes the config to generate default keystore by including /config/configDropins/defaults/keystore.xml. Liberty server generates /opt/ibm/wlp/output/defaultServer/resources/security/key.p12. Liberty server tries to load it. This is when the error occurs. I am not sure whether this is working as designed or if it's a bug. Please enable trace with the following traceSpecification and send the logs: *=info:SSL=all:SSLChannel=all

You have a couple of options to avoid the error:

  1. Modify tlsSecurity.xml to use defaultKeyStore as the id of keyStore and reference it in ssl config. This will override the defaultKeyStore from /config/configDropins/defaults/keystore.xml.
<server>
  <sslDefault sslRef="odmDefaultSSLConfig" />
  <ssl id="odmDefaultSSLConfig" keyStoreRef="defaultKeyStore" sslProtocol=.... />
  <keyStore id="defaultKeyStore" location=... />
</server>

I made the above change to the Pod manually and your keystore was loaded:

com.ibm.ws.ssl.config.WSKeyStore                             I Successfully loaded default keystore: /config/security/volume/pkcs11cfg.cfg of type: PKCS11-NSS-FIPS

Due to the override, you'll see the following in the logs which informs that the config in tlsSecurity.xml will be used. This is expected.

com.ibm.ws.config.xml.internal.ConfigValidator               A CWWKG0102I: Found conflicting settings for defaultKeyStore instance of keyStore configuration.
  Property password has conflicting values:
    Secure value is set in file:/opt/ibm/wlp/usr/servers/defaultServer/configDropins/defaults/keystore.xml.
    Secure value is set in file:/opt/ibm/wlp/usr/servers/defaultServer/tlsSecurity.xml.
  Property password will be set to the value defined in file:/opt/ibm/wlp/usr/servers/defaultServer/tlsSecurity.xml.
  1. Alternatively, you could copy an empty server config file to /config/configDropins/defaults/keystore.xml in your application Dockerfile (prior to RUN configure.sh if it's invoked). Liberty image will skip including config to generate default keystore since the file already exists. We'll consider adding a build argument instead to skip generating this default keystore.

keystore.xml :

<server/>

Dockerfile :

COPY --chown=1001:0 keystore.xml /config/configDropins/defaults/keystore.xml

I understand that the recommended configuration has been tested but currently fails within the ODM image.

mmouly commented

No, that's ok. thanks @leochr I tested the workaround and "parseAlgParameters failed: PBES2 AlgorithmParameters not available" is gone. I just wait for the validation as I still cannot have my apps available because of an other problem "No cipher list in common". I still don't know if the issue is on liberty and/or OCP side

@leochr did you see the comments above ? any way to unblock us ?

leochr commented

Sorry, I am not familiar with the new error: "No cipher list in common"

Most likely it's unrelated to containers, so best looked at by the Liberty runtime/security folks. Suggest opening a support case and provide more details on when this occurs, server logs, etc

mmouly commented

many thanks @leochr , I'm closing this ticket as workaround is working and the other issue "No cipher list in common" has no relation with this one