Add RabbitMQ as an event source
sc68cal opened this issue · 10 comments
Please confirm the following
- I agree to follow this project's code of conduct.
- I have checked the current issues for duplicates.
- I understand that ansible-rulebook is open source software provided for free and that I might not receive a timely response.
Feature type
New Feature
Feature Summary
It would be great to have RabbitMQ as an event source
Hello @sc68cal
After careful consideration, we have concluded that a plugin for RabbitMQ does not align with the scope of our official collection. For your reference, you may find this pull request insightful.
@Alex-Izquierdo can you share why this does not align with the scope of this official collection? Two separate contributors have created PRs that implemented this feature, so it is in demand.
Hi @sc68cal
Eda-collection is intended to provide the initial resources for getting started with the eda-server, as well as the integrations required for its features, by delegating third-party integrations to the collections developed by the ansible community.
@Alex-Izquierdo I do not agree with your conclusion. Discoverability on Ansible Galaxy leaves much to be desired, and I don't believe that creating a whole new Ansible galaxy collection to host a single 150 line python file is really the way forward with this. Especially when it is tied very closely to event driven ansible itself.
I also am disappointed by the lack of communication and how brusquely this issue appears to be handled, since it seems that there's no room for discussion or understanding of other people's viewpoints on this matter. I wish you would be more transparent and open to discussion, before making these decisions.
@sc68cal I'm really sorry for having seemed abrupt. That was never my intention and I should not have closed this issue. Indeed we are open to every discussion or suggestion from the community. I hope you will disregard this unfortunate mistake.
I understand your point of view and I think it's quite reasonable. Keep in mind that with each plugin added to this collection, there is a commitment to maintenance and therefore an associated effort. As I have previously explained, the purpose of this collection is to provide the basic set of tools for starting with eda-server and those integrations necessary for its functionality. It is not our intention to turn this collection into an all-in-one that contains all potential plugins, nor do we have the resources to maintain such a set.
Indeed, I think you are right that creating a single collection for a plugin can be overkill. Instead, I can suggest that you propose this extension to the rabbitmq community collection that already exists, which would also be quite coherent in terms of making accessible a collection that provides all the necessary tools for integration with the ansible ecosystem and rabbitmq (modules, eda extensions, etc.). That is why eda plugins are implemented as an extension inside collections. If users intend to use rabbitmq as a source of events for eda, it makes sense that they would also want to perform other configurations on queues for example, so it would be logical to do everything from the same collection.
Another option that we considered some time ago was to promote the creation of a plugin collection for eda plugins exclusively maintained by the community, something equivalent to community.general. We eventually discarded it because at that time there didn't seem to be enough traction or contributions to justify its existence. But I am sure that it is an option that can be reconsidered if the situation changes. Still, we continue to have the doubt of what is more practical and intuitive for users. Whether a repository that aggregates plugins of various natures is more consistent, or if it is more consistent for each collection to have the eda plugins related to the scope of that collection (a rabbitmq collection, a kubernetes collection, etc.)
indeed, I think you are right that creating a single collection for a plugin can be overkill. Instead, I can suggest that you propose this extension to the rabbitmq community collection that already exists, which would also be quite coherent in terms of making accessible a collection that provides all the necessary tools for integration with the ansible ecosystem and rabbitmq (modules, eda extensions, etc.).
In theory this would be acceptable, however the rabbitmq community collection uses pika as its RabbitMQ library, which does not work with asyncio. I would be adding a brand new dependency to aiorabbit which could turn into a large bikeshedding debate.
In addition, it would be easy for the rabbitmq community collection maintainers to turn around and use the same argument as yours, where dealing with event driven ansible is out of scope for their collection, and why isn't it in the EDA collection?
Basically, I think that for now, EDA is the location where this should go, since it would co-exist quite well with the other event sources that are being maintained.
In addition, I see that a PostgreSQL event source has been proposed in another PR, but I do not see the same pushback there, where the event source plugin should be sent over to the community.postgresql collection.
Can you help shed light on this situation for me?
Hi @sc68cal
I'm not aware of the particularities of each collection and I assume that including an eda plugin in some existing collection would require a discussion with their maintainers. I'm just suggesting reasonable alternatives to not maintain your own collection for this plugin.
I understand that some of the existing plugins in this collection might create confusion because they would not fit as well in the scope of the collection but this is because were added in the earliest stages of the development before this decision was taken and we just keep them there for backward compatibility. They will be eventually deprecated at some point.
In the case of your example, I presume you are talking about #181, is a plugin required for one of the new features of eda-server that we are currently working on, so it perfectly goes into our criteria.
@Alex-Izquierdo Which standards-conforming event distribution machanism could be considered basic enough to be considered for inclusion into and to be supported by eda.
mqtt?
There seems to be a way to create an integration test.
We need a sign, that eda has a valid scope outside product release cycles. Sitting outside with webhooks does not give me the feeling of participation either.
Hello @fkuep
I think it's not just a matter of determining which mechanism is basic enough to be included (I agree that MQTT is definitely one of the most relevant ones). The main point concerns the purpose and scope of this collection, which I explained in previous comments.
Hello, @Alex-Izquierdo I understand and comming from the rulebook doc I was of the impression :
- This is the repository one starts to learn with.
- one should best start with "event-bus-like event distribution systems" for activating rulebooks.
- the new and shiny event driven ansible thingy is going to be big also for the ansible community.
What I want from the Team working on this repository is either to develop an outline, how this can be used as independant reference implementation or ..
Have the ansible-rulebook documentation changed to reflect that:
This eda repo is a component of IBM Cloud offerings, we have this sandbox and we don´t share the shovel. Thanks for Your bug-fixes!
If it is the first one, one event mechanism that You consider best-practise should be
- part of this or an other reference implementation repository,
- fairly easy to be installed (e.g. not three nodes, zookeeper and java)
- Standards conformant
- tested with ci.
- kept around , at least until other low-effort/high quality reference event sources are available.
So please Alex ask around for us: We would like to know, if we are invited to use and promote this or if we are better off using https://github.com/adnanh/webhook to avoid technical dept.