phutchins/logstash-input-mongodb

This project is dead?

Jose92 opened this issue · 6 comments

Jose92 commented

Hello this project is dead?

matiasperrone commented

It seems that way...

My workaround was to use mongosql or a JDBC driver from DbSchema (check: https://dbschema.com/jdbc-driver/MongoDb.html).
Notice that mongosql generates "subtables" for the array fields and you can consult using SQL 92.

I prefered the JDBC driver due to the fact that almost any shell query or function would work, like: db.MyCollection.find()

I hope it helps.

  • 👍4
hardiksondagar commented

@matiasperrone I was looking for ways to sync data between mongodb to elasticsearch. The current plugin do most of basics stuff, but it does not support synching update/deletion of documentation. While other options seems deprecated for ES7.x, I also found logstash with JDBC driver looks most promising.

hardiksondagar commented

After spending hours figuring out of data pipeline, I end up writing custom python script to ingest data from mongodb to elasticsearch https://github.com/hardiksondagar/mongodb-to-elasticsearch

matiasperrone commented

@hardiksondagar seems that the new 7.9 has a beats plugin, so seems what are you looking for.
Check this out https://www.elastic.co/es/blog/whats-new-elastic-observability-7-9-0-unified-agent-kpi-overview-dashboard

sergio11 commented

Using JDBC driver from DbSchema is the better option, I have shared in this post my configuration in case it can help someone with this same problem, Regards.

  • 👍4
vinirib commented

I managed that with this updated infos using jdbc connection.