Salesforce Pipeline aborted due to error in logstash

Question

Salesforce Pipeline aborted due to error in logstash

nareshahi opened this issue 4 years ago · 6 comments

Hi All, Please help if any one successfully integrated Salesforce cloud logs in SIEM.
I am using the following config pipeline file in logstash to fetch the logs from cloud. but Pipeline aborted due to following error.

Kindly help me how i can overcome this issue.

Config File (if you have sensitive info, please remove it):

input {
salesforce {
client_id => "yyyyyy"
client_secret => "Aaaaaaaa"
username => ""
password => ""
security_token => ""
sfdc_object_name => "uuuuuu"
}
}

filter {
mutate {
rename => ["host", "host.ip"]
}
}

output {

stdout { codec => rubydebug { metadata => true } }

lumberjack {
hosts => ["xxxxxx"]
codec => json
ssl_certificate => "/etc/logstash/certs/logstash.cert"
port => xxxx
}
}

Error

[2020-10-14T10:36:59,350][ERROR][logstash.agent ] Failed to execute action {:id=>:salesforce, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}
[2020-10-14T10:37:11,399][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"salesforce", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x232bc62e run>"}
[2020-10-14T10:38:12,339][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"salesforce", :exception=>#Faraday::ConnectionFailed, :backtrace=>["org/jruby/ext/socket/RubyTCPSocket.java:119:in initialize'", "org/jruby/RubyIO.java:1155:in open'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:941:in block in connect'", "org/jruby/ext/timeout/Timeout.java:99:in timeout'", "org/jruby/ext/timeout/Timeout.java:75:in timeout'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:939:in connect'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in do_start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:913:in start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1465:in request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:82:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:40:in block in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:87:in with_net_http_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:32:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response_middleware.rb:31:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/request/url_encoded.rb:15:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:in run_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:177:in post'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:24:in authenticate!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:18:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/request/encode_json.rb:24:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/multipart.rb:16:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:in run_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:140:in get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:37:in block in define_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:63:in block in define_api_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/api.rb:128:in describe'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb:90:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in block in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:280:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:145:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:104:in block in start'"], :thread=>"#<Thread:0x232bc62e run>"}
[2020-10-14T10:38:12,355][ERROR][logstash.agent ] Failed to execute action {:id=>:salesforce, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}
[2020-10-14T10:38:14,459][INFO ][logstash.javapipeline ] Starting pipeline {:pipeline_id=>"salesforce", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, :thread=>"#<Thread:0x1c592dfa run>"}

Answer 1 · 2020-10-15T05:34:49.000Z

Hello Folks, please help if anyone face this issue.

Answer 2 · 2020-10-15T06:50:52.000Z

Hi nareshahi,

you an authentication failure (as you already may have figured out).
This might be due to the missing salesforce input field: "api_version".
Add it to your salesforce input config and see if that helps.
In my case it is: api_version => "43.0"
Regards,
Martin.

Answer 3 · 2020-10-19T23:50:09.000Z

Thanks Martin. Tried with your provided input but it is still same suition.

[2020-10-20T10:47:08,602][ERROR][logstash.javapipeline ] Pipeline aborted due to error {:pipeline_id=>"salesforce", :exception=>#Faraday::ConnectionFailed, :backtrace=>["org/jruby/ext/socket/RubyTCPSocket.java:119:in initialize'", "org/jruby/RubyIO.java:1155:in open'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:941:in block in connect'", "org/jruby/ext/timeout/Timeout.java:99:in timeout'", "org/jruby/ext/timeout/Timeout.java:75:in timeout'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:939:in connect'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:924:in do_start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:913:in start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1465:in request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:82:in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:40:in block in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:87:in with_net_http_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/adapter/net_http.rb:32:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response_middleware.rb:31:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/request/url_encoded.rb:15:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:in run_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:177:in post'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:24:in authenticate!'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/authentication.rb:18:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/request/encode_json.rb:24:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/multipart.rb:16:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/middleware/mashify.rb:8:in call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:in build_response'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:377:in run_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.9.2/lib/faraday/connection.rb:140:in get'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:37:in block in define_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:63:in block in define_api_verb'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/api.rb:128:in describe'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb:90:in register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:191:in block in register_plugins'", "org/jruby/RubyArray.java:1792:in each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:190:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:280:in start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:244:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:145:in run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:104:in block in start'"], :thread=>"#<Thread:0x33bc5a4f run>"}

Answer 4 · 2021-01-20T04:11:45.000Z

Hello Martin,
you mentioned in my case, can you please provide you Salesforce.conf so that i would map if anything missing.

regards
nareshahi

Answer 5 · 2021-01-27T01:49:13.000Z

Hi Martin, i am have proxy to reach salesforce cloud.
please suggest what setting i need to do on .conf pipeline to reach slaesforce via proxy.

Answer 6 · 2021-11-10T09:34:31.000Z

@nareshahi have you seen https://github.com/logstash-plugins/logstash-input-salesforce/blob/main/docs/index.asciidoc#http-proxy which describes how to configure a proxy for the connection to Salesforce? Does this solve your problem?