Connection to Nifi only succeeds locally (`nipyapi.canvas.get_root_pg_id` API call)
andyadamides opened this issue · 9 comments
andyadamides commented
When I execute the following from my local session:
import nipyapi
nipyapi.config.nifi_config.verify_ssl = False
nipyapi.config.nifi_config.host = "<HOST>/nifi-api"
nipyapi.utils.set_endpoint("<HOST>/nifi-api")
# Queues can be large - increase timeout
nipyapi.config.short_max_wait = 3600
nifi_token_http_info = nipyapi.nifi.AccessApi().create_access_token_with_http_info(
username="<Username>",
password="<Password",
)
nipyapi.config.nifi_config.api_key["<Username>"] = nifi_token_http_info[0]
nipyapi.config.nifi_config.api_client = nipyapi.nifi.ApiClient(
header_name="Authorization",
header_value="Bearer {}".format(nifi_token_http_info[0]),
cookie=nifi_token_http_info[2]["set-cookie"],
)
nipyapi.canvas.get_root_pg_id()It always succeeds, but when moving this same process in a set of CI Servers it randomly fails(sometimes it works, sometimes it doesn't) with:
File "/usr/lib/python3.7/site-packages/nipyapi/canvas.py", line 44, in get_root_pg_id
return nipyapi.nifi.FlowApi().get_process_group_status('root') \
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/apis/flow_api.py", line 2833, in get_process_group_status
(data) = self.get_process_group_status_with_http_info(id, **kwargs)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/apis/flow_api.py", line 2923, in get_process_group_status_with_http_info
collection_formats=collection_formats)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/api_client.py", line 334, in call_api
_return_http_data_only, collection_formats, _preload_content, _request_timeout)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/api_client.py", line 155, in __call_api
_request_timeout=_request_timeout)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/api_client.py", line 357, in request
headers=headers)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/rest.py", line 244, in GET
query_params=query_params)
File "/usr/lib/python3.7/site-packages/nipyapi/nifi/rest.py", line 235, in request
raise ApiException(http_resp=r)
nipyapi.nifi.rest.ApiException: (401)
Reason: Unauthorized
error_description="An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found", error_uri="https://tools.ietf.org/html/rfc[67](https://gitlab.com/williamhillplc/data/data-services/smart-data-platform-logical/-/jobs/3131572537#L67)50#section-3.1"', 'Server': 'Jetty(9.4.46.v20220331)'})The user-log has this when the above occurs:
2022-10-05 15:38:48,529 ERROR [NiFi Web Server-22] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: org.springframework.security.oauth2.server.resource.InvalidBearerTokenException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found. Returning Internal Server Error response.
org.springframework.security.oauth2.server.resource.InvalidBearerTokenException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found
at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.getJwt(JwtAuthenticationProvider.java:101)
at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.authenticate(JwtAuthenticationProvider.java:88)
at org.apache.nifi.web.api.AccessResource.getAccessStatus(AccessResource.java:252)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1459)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:204)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.filter.ExceptionFilter.doFilter(ExceptionFilter.java:46)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:487)
at org.apache.nifi.web.server.filter.DataTransferExcludedDoSFilter.doFilterChain(DataTransferExcludedDoSFilter.java:51)
at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:336)
at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:301)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.server.log.RequestAuthenticationFilter.doFilterInternal(RequestAuthenticationFilter.java:59)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.StrictTransportSecurityFilter.doFilter(StrictTransportSecurityFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XContentTypeOptionsFilter.doFilter(XContentTypeOptionsFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XSSProtectionFilter.doFilter(XSSProtectionFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.ContentSecurityPolicyFilter.doFilter(ContentSecurityPolicyFilter.java:47)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.apache.nifi.web.security.headers.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:48)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.springframework.security.oauth2.jwt.BadJwtException: An error occurred while attempting to decode the Jwt: Signed JWT rejected: Another algorithm expected, or no matching key(s) found
at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.createJwt(NimbusJwtDecoder.java:180)
at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.decode(NimbusJwtDecoder.java:137)
at org.springframework.security.oauth2.server.resource.authentication.JwtAuthenticationProvider.getJwt(JwtAuthenticationProvider.java:97)
... 104 common frames omitted
Caused by: com.nimbusds.jose.proc.BadJOSEException: Signed JWT rejected: Another algorithm expected, or no matching key(s) found
at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:357)
at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:303)
at org.springframework.security.oauth2.jwt.NimbusJwtDecoder.createJwt(NimbusJwtDecoder.java:154)
... 106 common frames omittedAre there any ideas as to what is going on or how to fix this?
Chaffelson commented
Is your CI session possibly working with a cluster of nodes and not a single node? This looks suspiciously like the errors you get when a token for cluster node-x is presented to cluster node-y by mistake.
Also, why are you manually setting up the bearer token instead of using the login optional functionality of set_endpoint?
andyadamides commented
Yes the issue relates to having a cluster setup rather than a single node: https://issues.apache.org/jira/browse/NIFI-10606
andyadamides commented
Also, why are you manually setting up the bearer token instead of using the login optional functionality of set_endpoint?
Are you suggesting to use service_login ? (https://nipyapi.readthedocs.io/en/latest/nipyapi-docs/nipyapi.html#nipyapi.security.service_login)
Chaffelson commented
I find the easiest solution is nipyapi.utils.set_endpoint(uri, ssl, login, user, password)
If this doesn't work for your case I'd be keen to know the specifics to see if we can make you a better connection utility function - but set_endpoint is 100% intended to be a convenience method for this.
andyadamides commented
@Chaffelson
This works:
curl --cacert "ca.crt" --cert "tls.crt" --key "tls.key" https://<host>:8443/nifi-api/access -v -k
Also, this works:
import requests
cert = ('tls.crt', 'tls.key')
response = requests.get('https://<host>:8443/nifi-api/access', cert=cert, verify=False)
print(response.json())But the following doesn't:
nipyapi.config.nifi_config.verify_ssl = False
nipyapi.security.set_service_ssl_context(
service='nifi',
ca_file='ca.crt',
client_cert_file='tls.crt',
client_key_file='tls.key',
)
nipyapi.utils.set_endpoint("https://<host>:8443/nifi-api", ssl=True, login=False,
username=None, password=None)
nipyapi.security.get_service_access_status("nifi")It gives me
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<host>', port=8443): Max retries exceeded with url: /nifi-api/access (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1091)')))
I went through the code of set_endpoint, it eventually calls set_service_ssl_context ; I have tried various permutations but I cannot get it work with nipyapi while it works like a charm with curl - what do you suggest trying out here?
andyadamides commented
After some fiddling around, turns out I have had to do it this way in order to work:
import nipyapi
nipyapi.config.nifi_config.verify_ssl = False
nipyapi.config.default_ssl_context = {
'ca_file':'ca.crt',
'client_cert_file':'tls.crt',
'client_key_file':'tls.key',
}
nipyapi.utils.set_endpoint("https://<host>:8443/nifi-api", ssl=True, login=False,
username=None, password=None)
nipyapi.security.get_service_access_status("nifi")Chaffelson commented
It may be a good idea if we update the documentation here with this specific example to help out others in future, as it's not in the secure_connection demo and it obviously seems a relevant usecase.
Is there any particular reason for verify_ssl false? Is it just a self-signed certificate?
andyadamides commented
@Chaffelson Which part of the documentation would you suggest to edit, I can prepare it?
i.e. here?
Chaffelson commented
Yeah giving a better docstring with explanations would vastly improve usage - if you want to prepare it that would be much appreciated ^-^