Problems loading a plugin - Google BigQuery
Opened this issue · 0 comments
Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.
For all general issues, please provide the following details for fast resolution:
- Version: 7.3.1
- Operating System: Windows 10
- Config File (if you have sensitive info, please remove it):
input {
file {
path => "full-path
/logstash-tutorial-dataset"
sincedb_path => "nul"
start_position => "beginning"
}
}
filter {
grok {
match => {
"message" => "%{COMBINEDAPACHELOG}"
#'message' => '%{IP:ip} - - [%{GREEDYDATA:date}] %{GREEDYDATA:request}'
}
}
geoip {
source => "clientip"
}
}
output {
google_bigquery {
project_id => "akbm-infrastructur"
dataset => "test_logstash"
csv_schema => "clientip:STRING"
json_key_file => "full-path-to-key-file
.json"
error_directory => "/tmp/bigquery-errors"
}
}
-
Sample Data:
83.149.9.216 - - [04/Jan/2015:05:13:42 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36" -
Steps to Reproduce:
Running the config file with `bin/logstash -r -f “” with the purpose of sending local data to BigQuery, using Logstash, produces a log file consisting of mainly
[2019-08-29T09:47:18,692][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"output", :name=>"google_bigquery", :path=>"logstash/outputs/google_bigquery", :error_message=>"\n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command\n\nno such file to load -- com/google/cloud/google-cloud-bigquery/1.24.1/google-cloud-bigquery-1.24.1 (LoadError)", :error_class=>RuntimeError, :error_backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:356:in `do_require'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:265:in `block in require_jar'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:307:in `require_jar_with_block'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:264:in `require_jar'",
And
[2019-08-29T09:47:18,697][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any output plugin named 'google_bigquery'. Are you sure this is correct? Trying to load the google_bigquery output plugin resulted in this error: Problems loading the requested plugin named google_bigquery of type output. Error: RuntimeError \n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command\n\nno such file to load -- com/google/cloud/google-cloud-bigquery/1.24.1/google-cloud-bigquery-1.24.1 (LoadError)", :backtrace=>
Please note that when running bin/logstash-plugin list
it outputs logstash-output-google_bigquery