When server is being asked lots of queries under heavy load a single search on top returns zero results
smokingwheels opened this issue · 1 comments
start up settings.
JAVA_ARGS="-XX:+UseG1GC -XX:MaxGCPauseMillis=400 -XX:G1HeapRegionSize=64m -XX:MetaspaceSize=1024m -XX:MaxMetaspaceSize=2048m -Xss2048k $JAVA_ARGS";
I have 300 GB assigned.
My Yacy server is under a search load of 15 Hz.
`I 2024/08/30 19:56:09 NETWORK * SEARCH failed (solr), localpeer (Error executing query) java.io.IOException: Error executing query at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:376) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getResponseByParams(MirrorSolrConnector.java:272) at net.yacy.peers.Protocol.solrQuery(Protocol.java:1225) at net.yacy.peers.RemoteSearch$2.run(RemoteSearch.java:411) Caused by: org.apache.solr.common.SolrException: Error while processing facet fields: InterruptedException at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:936) at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:337) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:278) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:420) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:207) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2866) at net.yacy.cora.federate.solr.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:231) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:217) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:927) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:940) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:369) ... 3 more Caused by: java.lang.InterruptedException at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1100) at java.base/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:925) ... 13 more
E 2024/08/30 19:56:09 org.apache.solr.handler.RequestHandlerBase org.apache.solr.common.SolrException: Error while processing facet fields: InterruptedException org.apache.solr.common.SolrException: Error while processing facet fields: InterruptedException at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:936) at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:337) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:278) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:420) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:207) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2866) at net.yacy.cora.federate.solr.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:231) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:217) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:927) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:940) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:369) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getResponseByParams(MirrorSolrConnector.java:272) at net.yacy.peers.Protocol.solrQuery(Protocol.java:1225) at net.yacy.peers.RemoteSearch$2.run(RemoteSearch.java:411) Caused by: java.lang.InterruptedException at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1100) at java.base/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:925) ... 13 more
I 2024/08/30 19:56:09 NETWORK * SEARCH failed (solr), localpeer (Error executing query) java.io.IOException: Error executing query at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:376) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getResponseByParams(MirrorSolrConnector.java:272) at net.yacy.peers.Protocol.solrQuery(Protocol.java:1225) at net.yacy.peers.RemoteSearch$2.run(RemoteSearch.java:411) Caused by: org.apache.solr.common.SolrException: Error while processing facet fields: InterruptedException at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:936) at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:337) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:278) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:420) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:207) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2866) at net.yacy.cora.federate.solr.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:231) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:217) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:927) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:940) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:369) ... 3 more Caused by: java.lang.InterruptedException at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1100) at java.base/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:925) ... 13 more
I 2024/08/30 19:55:30 LOCAL_SEARCH * EXIT WORD SEARCH: olqw - local_rwi_available(0), local_rwi_stored(0), remote_rwi_available(0), remote_rwi_stored(0), remote_rwi_peerCount(0), local_solr_evicted(0), local_solr_stored(0), remote_solr_available(0), remote_solr_stored(0), remote_solr_peerCount(0), 1344 ms
I 2024/08/30 19:56:09 NETWORK * SEARCH failed (solr), localpeer (Error executing query) java.io.IOException: Error executing query at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:376) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getResponseByParams(MirrorSolrConnector.java:272) at net.yacy.peers.Protocol.solrQuery(Protocol.java:1225) at net.yacy.peers.RemoteSearch$2.run(RemoteSearch.java:411) Caused by: org.apache.solr.common.SolrException: Error while processing facet fields: InterruptedException at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:936) at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:337) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:278) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:420) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:207) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2866) at net.yacy.cora.federate.solr.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:231) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:217) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:927) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:940) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getResponseByParams(EmbeddedSolrConnector.java:369) ... 3 more Caused by: java.lang.InterruptedException at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1100) at java.base/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:925) ... 13 more
I 2024/08/30 19:56:09 NETWORK * SEARCH failed (solr), localpeer (Error executing query) java.io.IOException: Error`
I stopped asking and yacy started responding to queries.
`I 2024/08/30 20:07:45 NETWORK * local search (solr): localpeer sent 10/11 references
I 2024/08/30 20:07:45 NETWORK * SEARCH (solr), returned 10 out of 11 documents and 5 facets [url_file_ext_s, author_sxt, url_protocol_s, host_s, language_s] from peer pX4MdxchuNOm:agent-ramdisk2
I 2024/08/30 20:07:45 NETWORK * local search (solr): localpeer sent 10/20 references
I 2024/08/30 20:07:45 NETWORK * SEARCH (solr), returned 10 out of 20 documents and 5 facets [url_file_ext_s, author_sxt, url_protocol_s, host_s, language_s] from peer pX4MdxchuNOm:agent-ramdisk2`