wso2/product-apim

WSO2 API Manager(wso2am-4.0.0) - Recurring JMS Error

LilanMihiranga opened this issue · 1 comments

Description:

I have setup wso2am-4.0.0 and I have configured readonly ldap as the primary userstore.

After changing to readonly ldap, the following error is seen in the logs and it is recurring forever. The API calls also seem to be failing due to it.

Steps to reproduce:

  1. Create an ldap user.
  2. Configure the deployment.toml to point the ldap as the primary read-only user store.
    deployment.toml
[user_store]
class="org.wso2.carbon.user.core.ldap.ReadOnlyLDAPUserStoreManager"
type = "read_only_ldap"

[user_store.properties]
TenantManager="org.wso2.carbon.user.core.tenant.CommonHybridLDAPTenantManager"
ConnectionURL="ldap://<ldap_ip>:<port>"
ConnectionName="<cn>"
ConnectionPassword="<password>"
AnonymousBind="false"
UserNameAttribute="<username_attribute>"
UserNameSearchFilter="(&amp;(objectClass=user)(sAMAccountName=?))"
ReadGroups="true"
GroupSearchBase="<search_base>
GroupNameAttribute="cn"
GroupNameSearchFilter="(&amp;(objectClass=groupOfNames)(cn=?))"
GroupNameListFilter="(objectClass=groupOfNames)"
MembershipAttribute="member"
BackLinksEnabled="false"
UsernameJavaRegEx="[a-zA-Z0-9._\\-|//]{3,30}$"
PasswordJavaRegEx="^[\\S]{5,30}$"
SCIMEnabled="true"
PasswordHashMethod="PLAIN_TEXT"
MultiAttributeSeparator=","
MaxUserNameListLength="100"
MaxRoleNameListLength="100"
UserRolesCacheEnabled="true"
LDAPConnectionTimeout=5000
ReplaceEscapeCharactersAtUserLogin="true"
ConnectionRetryDelay="120000"
GroupSearchFilter="(objectClass=groupOfNames)"
UserEntryObjectClass="identityPerson"
IsBulkImportSupported="true"
defaultRealmName="WSO2.ORG"
EmptyRolesAllowed="true"
UserSearchBase="<search_base>"
ConnectionPoolingEnabled="false"
StartTLSEnabled="false"
WriteGroups="true"
RolenameJavaRegEx="[a-zA-Z0-9._\\-|//]{3,30}$"
GroupEntryObjectClass="groupOfNames"
EnableMaxUserLimitForSCIM="false"
PasswordJavaRegExViolationErrorMsg="Password length should be within 5 to 30 characters"
PasswordJavaScriptRegEx="^[\\S]{5,30}$"
UsernameJavaRegExViolationErrorMsg="Username pattern policy violated"
UserNameListFilter="(objectClass=person)"
UsernameJavaScriptRegEx="^[\\S]{3,30}$"
kdcEnabled="false"

wso2carbon.log

TID: [-1] [] [2022-04-12 11:55:24,706]  INFO {org.wso2.carbon.apimgt.common.jms.JMSTaskManager} - Reconnection attempt: 1 for Siddhi-JMS-Consumer was successful!
TID: [-1] [] [2022-04-12 11:55:24,829] ERROR {org.wso2.carbon.apimgt.common.jms.JMSTaskManager} - Error creating JMS consumer for Siddhi-JMS-Consumer javax.jms.JMSException: Error registering consumer: org.wso2.andes.AMQChannelClosedException: Error: org.wso2.andes.AMQSecurityException: Permission denied: binding notification [error code 403: access refused] [error code 504: channel error]
        at org.wso2.andes.client.AMQSession$6.execute(AMQSession.java:2187)
        at org.wso2.andes.client.AMQSession$6.execute(AMQSession.java:2130)
        at org.wso2.andes.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:339)
        at org.wso2.andes.client.AMQConnection$3.run(AMQConnection.java:665)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.wso2.andes.client.AMQConnection.executeRetrySupport(AMQConnection.java:662)
        at org.wso2.andes.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
        at org.wso2.andes.client.AMQSession.createConsumerImpl(AMQSession.java:2195)
        at org.wso2.andes.client.AMQSession.createConsumer(AMQSession.java:1100)
        at org.wso2.carbon.apimgt.common.jms.utils.JMSUtils.createConsumer(JMSUtils.java:495)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.createConsumer(JMSTaskManager.java:1010)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.getMessageConsumer(JMSTaskManager.java:865)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.receiveMessage(JMSTaskManager.java:612)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.run(JMSTaskManager.java:533)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.andes.AMQChannelClosedException: Error: org.wso2.andes.AMQSecurityException: Permission denied: binding notification [error code 403: access refused] [error code 504: channel error]
        at org.wso2.andes.client.handler.ChannelCloseMethodHandler.methodReceived(ChannelCloseMethodHandler.java:100)
        at org.wso2.andes.client.handler.ClientMethodDispatcherImpl.dispatchChannelClose(ClientMethodDispatcherImpl.java:163)
        at org.wso2.andes.framing.amqp_0_91.ChannelCloseBodyImpl.execute(ChannelCloseBodyImpl.java:140)
        at org.wso2.andes.client.state.AMQStateManager.methodReceived(AMQStateManager.java:111)
        at org.wso2.andes.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:554)
        at org.wso2.andes.client.protocol.AMQProtocolSession.methodFrameReceived(AMQProtocolSession.java:467)
        at org.wso2.andes.framing.AMQMethodBodyImpl.handle(AMQMethodBodyImpl.java:92)
        at org.wso2.andes.client.protocol.AMQProtocolHandler$2.run(AMQProtocolHandler.java:487)
        at org.wso2.andes.pool.Job.processAll(Job.java:111)
        at org.wso2.andes.pool.Job.run(Job.java:158)
        ... 3 more

TID: [-1] [] [2022-04-12 11:55:24,832] ERROR {org.apache.axis2.transport.base.threads.NativeWorkerPool} - Uncaught exception org.wso2.carbon.apimgt.common.jms.JmsRunTimeException
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager.handleException(JMSTaskManager.java:1144)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager.access$800(JMSTaskManager.java:62)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.createConsumer(JMSTaskManager.java:1019)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.getMessageConsumer(JMSTaskManager.java:865)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.receiveMessage(JMSTaskManager.java:612)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.run(JMSTaskManager.java:533)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

TID: [-1] [] [2022-04-12 11:55:24,834] ERROR {org.wso2.carbon.apimgt.common.jms.JMSTaskManager} - JMS Connection failed : 504 - shutting down worker tasks
TID: [-1] [] [2022-04-12 11:55:24,834]  INFO {org.wso2.carbon.apimgt.common.jms.JMSTaskManager} - Reconnection attempt : 1 for Siddhi-JMS-Consumer
TID: [-1] [] [2022-04-12 11:55:25,023] ERROR {org.wso2.carbon.apimgt.common.jms.JMSTaskManager} - Error creating JMS consumer for Siddhi-JMS-Consumer javax.jms.JMSException: Error registering consumer: org.wso2.andes.AMQChannelClosedException: Error: org.wso2.andes.AMQSecurityException: Permission denied: binding notification [error code 403: access refused] [error code 504: channel error]
        at org.wso2.andes.client.AMQSession$6.execute(AMQSession.java:2187)
        at org.wso2.andes.client.AMQSession$6.execute(AMQSession.java:2130)
        at org.wso2.andes.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:339)
        at org.wso2.andes.client.AMQConnection$3.run(AMQConnection.java:665)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.wso2.andes.client.AMQConnection.executeRetrySupport(AMQConnection.java:662)
        at org.wso2.andes.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
        at org.wso2.andes.client.AMQSession.createConsumerImpl(AMQSession.java:2195)
        at org.wso2.andes.client.AMQSession.createConsumer(AMQSession.java:1100)
        at org.wso2.carbon.apimgt.common.jms.utils.JMSUtils.createConsumer(JMSUtils.java:495)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.createConsumer(JMSTaskManager.java:1010)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.getMessageConsumer(JMSTaskManager.java:865)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.receiveMessage(JMSTaskManager.java:612)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.run(JMSTaskManager.java:533)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.wso2.andes.AMQChannelClosedException: Error: org.wso2.andes.AMQSecurityException: Permission denied: binding notification [error code 403: access refused] [error code 504: channel error]
        at org.wso2.andes.client.handler.ChannelCloseMethodHandler.methodReceived(ChannelCloseMethodHandler.java:100)
        at org.wso2.andes.client.handler.ClientMethodDispatcherImpl.dispatchChannelClose(ClientMethodDispatcherImpl.java:163)
        at org.wso2.andes.framing.amqp_0_91.ChannelCloseBodyImpl.execute(ChannelCloseBodyImpl.java:140)
        at org.wso2.andes.client.state.AMQStateManager.methodReceived(AMQStateManager.java:111)
        at org.wso2.andes.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:554)
        at org.wso2.andes.client.protocol.AMQProtocolSession.methodFrameReceived(AMQProtocolSession.java:467)
        at org.wso2.andes.framing.AMQMethodBodyImpl.handle(AMQMethodBodyImpl.java:92)
        at org.wso2.andes.client.protocol.AMQProtocolHandler$2.run(AMQProtocolHandler.java:487)
        at org.wso2.andes.pool.Job.processAll(Job.java:111)
        at org.wso2.andes.pool.Job.run(Job.java:158)
        ... 3 more

TID: [-1] [] [2022-04-12 11:55:25,025] ERROR {org.apache.axis2.transport.base.threads.NativeWorkerPool} - Uncaught exception org.wso2.carbon.apimgt.common.jms.JmsRunTimeException
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager.handleException(JMSTaskManager.java:1144)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager.access$800(JMSTaskManager.java:62)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.createConsumer(JMSTaskManager.java:1019)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.getMessageConsumer(JMSTaskManager.java:865)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.receiveMessage(JMSTaskManager.java:612)
        at org.wso2.carbon.apimgt.common.jms.JMSTaskManager$MessageListenerTask.run(JMSTaskManager.java:533)
        at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Affected Product Version:

wso2am-4.0.0

Environment details (with versions):

  • OS: Linux

Used WSO2 Identity Server 5.11 to Create a LDAP user. In order to run both APIM and IS products locally, add port offset as 1 in IS. After that, followed this to configure a read only LDAP user store. Similar to above [user_store] and [user_store.properties]. Tested with both APIM 4.0 and 4.1 and was unable to reproduce the error log.