Fix or Replace Trellis use of ActiveMQ
Closed this issue · 3 comments
In the current Sinopia stack, the Trellis LDP publishes messages to an AWS ActiveMQ queue that is then picked up by the Indexing Pipeline that then either adds/updates/deletes the resource document in the Elasticsearch cluster. Occasionally for the past year-and-a-half, Trellis stop connecting to ActiveMQ which halts all Elasticsearch indexing activity. Trellis now supports AMQP or an alternative is to use Camel for direct indexing into Elasticsearch.
In the current Sinopia stack, the Trellis LDP publishes messages to an AWS ActiveMQ queue that is then picked up by the Indexing Pipeline that then either adds/updates/deletes the resource document in the Elasticsearch cluster. Occasionally for the past year-and-a-half, Trellis stop connecting to ActiveMQ which halts all Elasticsearch indexing activity. Trellis now supports AMQP or an alternative is to use Camel for direct indexing into Elasticsearch.
I have no attachment to ActiveMQ but I am wondering if we have a sense of what causes the disconnects. My worry is that we put effort into swapping out ActiveMQ for an AMQP-based back-end (or Camel) only to have them suffer from the same disconnects.
ActiveMQ connections are notoriously flacky because they maintain an active connection. In other words, I think the problem is with the ActiveMQ stack (exasperated by running in AWS, where the network is a black box), not anything Trellis specific.
That being said, nothing to guarantee won't encounter problems with AMQP or Camel. The Camel connector for ElasticSearch might be worth a look, as at least for communicating with ES, is straight HTTP.
Closing as Trellis is being depreciated in favor of MongoDB which doesn't use ActiveMQ.