cloudfoundry-community/logsearch-boshrelease

Throttle plugin available offline?

peterellisjones opened this issue · 2 comments

Hi release devs,

I'm part of a team using this release in an offline environment. We would like to use the logstash throttle filter plugin and need to choose one of two possible implementations:

  1. Create our own custom release that makes the plugin available on the ingestor VM and install with the plugins property
  2. Fork this release and install the plugin as part of the packaging script.

We would prefer to do 2) but don't want to maintain our own fork. Would you accept a PR from us making the throttle plugin available offline or would you consider that out of scope for this release? Our use-case is for throttling talkative Cloud Foundry applications.

please ignore this — just found out that the plugin is already available in the logstash package

Hi @peterellisjones! Just for the reference I would suggest the following workflow for plugins that are not available out-of-the-box:

  1. Download missing plugin
  2. Create a directory, put plugin into the directory and add plugin to zip archive:
    mkdir logstash_cache
    mv logstash-filter-throttle-4.0.4.gem logstash_cache
    cd logstash_cache
    zip logstash-filter-throttle-4.0.4.zip logstash-filter-throttle-4.0.4.gem
    rm logstash-filter-throttle-4.0.4.gem
  3. Create a file manifest.yml with syntax like below:
    vim manifest.yml 
    ---
    applications:
    - name: logstash_cache
      memory: 64M
      path: .
      buildpacks: 
      - staticfile_buildpack
  4. Push the application to any space on Cloud Foundry within a Logsearch foundation.
  5. Add opsfile with missing plugin to logsearch deployment:
    ---
    - type: replace
      path: /instance_groups/name=ingestor/jobs/name=ingestor_syslog/properties/logstash/plugins?
      value:
      - {name: http://logstashcache.apps.pcf.com/logstash-filter-throttle-4.0.4.zip}