A connection error occurs after TLS authentication is enabled
Howie516 opened this issue · 5 comments
When testing TLS, copy CA-related files in the server configs to the local PC, and apply after modifying the k8s configmap. After calling the SDK, an error was reported, and the network ping was successful
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 000006040000000000000500004000
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1213) ~[netty-handler-4.1.72.Final.jar:4.1.72.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1283) ~[netty-handler-4.1.72.Final.jar:4.1.72.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) ~[netty-codec-4.1.72.Final.jar:4.1.72.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) ~[netty-codec-4.1.72.Final.jar:4.1.72.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) ~[netty-transport-4.1.72.Final.jar:4.1.72.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[netty-common-4.1.72.Final.jar:4.1.72.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.72.Final.jar:4.1.72.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.72.Final.jar:4.1.72.Final]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]
After restarting pod, configmap appears to work, but you can't log in with your user name and password, and an error occurs when you log in with sdk tls:
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:146) ~[?:1.8.0_382] at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:127) ~[?:1.8.0_382] at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) ~[?:1.8.0_382] at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451) ~[?:1.8.0_382] at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323) ~[?:1.8.0_382] at sun.security.validator.Validator.validate(Validator.java:271) ~[?:1.8.0_382] at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315) ~[?:1.8.0_382] at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:278) ~[?:1.8.0_382] at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141) ~[?:1.8.0_382] at sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1334) ~[?:1.8.0_382] at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231) ~[?:1.8.0_382] at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174) ~[?:1.8.0_382] at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377) ~[?:1.8.0_382] at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) ~[?:1.8.0_382] at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:981) ~[?:1.8.0_382] at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:968) ~[?:1.8.0_382] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_382] at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:915) ~[?:1.8.0_382] at io.netty.handler.ssl.SslHandler$SslTasksRunner.run(SslHandler.java:1785) ~[netty-handler-4.1.72.Final.jar:4.1.72.Final] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_382] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_382] at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]
Follow the description of TLS example:
The following steps are an example of using docker-compose to launch a milvus cluster locally with tls configurations.
-
Download this zip file to a local folder, and extract it. There are docker-compose.yaml, milvus.yaml and a "tls" folder in it.
233.zip -
cd into the "tls" folder, generate the certification files
chmod +x gen.sh
./gen.sh
-
cd to the folder extracted by the 233.zip
docker-compose up -d
you will see a local cluster is started -
use java sdk to connect the server
ConnectParam connectParam = ConnectParam.newBuilder()
.withHost("localhost")
.withPort(19530)
.withServerName("localhost")
.withServerPemPath("[the extracted folder path]/tls/server.pem")
.build();
MilvusServiceClient milvusClient = new MilvusServiceClient(connectParam);
R<CheckHealthResponse> health = milvusClient.checkHealth();
if (health.getStatus() != R.Status.Success.getCode()) {
throw new RuntimeException(health.getMessage());
} else {
System.out.println(health);
}
The key points:
- In the docker-compose.yaml, the milvus.yaml is mapped to the ourside milvus.yaml, the internal certification file path "/milvus/configs/cert" is mapped to the outside "tls" folder
proxy:
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/milvus.yaml:/milvus/configs/milvus.yaml
- ${DOCKER_VOLUME_DIRECTORY:-.}/tls:/milvus/configs/cert
- In the outside milvus.yaml, tls paths and tls mode is configurated:
common:
security:
tlsMode: 1 # 1 is one-way tls
tls:
serverPemPath: /milvus/configs/cert/server.pem
serverKeyPath: /milvus/configs/cert/server.key
caPemPath: /milvus/configs/cert/ca.pem
- In the client java code, the correct file is specified:
ConnectParam connectParam = ConnectParam.newBuilder()
.withHost("localhost")
.withPort(19530)
.withServerName("localhost")
.withServerPemPath("[the extracted folder path]/tls/server.pem")
If you need two-way tls:
- set the tlsMode to 2 in the milvus.yaml:
common:
security:
tlsMode: 2
- specify the required certification files in the client java code
ConnectParam connectParam = ConnectParam.newBuilder()
.withHost("localhost")
.withPort(19530)
.withServerName("localhost")
.withCaPemPath("[the extracted folder path]/tls/ca.pem")
.withClientKeyPath("[the extracted folder path]/tls/client.key")
.withClientPemPath("[the extracted folder path]/tls/client.pem")
.build();
MilvusServiceClient milvusClient = new MilvusServiceClient(connectParam);