Index to elasticsearch not possible
rkscon opened this issue · 5 comments
Hello,
i encounter the following issue out of the logstash log:
[2019-01-18T11:53:18,482][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snort-1.0.0-2019.01.18", :_type=>"doc", :routing=>nil}, #LogStash::Event:0x7292b14b], :response=>{"index"=>{"_index"=>"snort-1.0.0-2019.01 .18", "_type"=>"doc", "_id"=>"UYXOYGgB9Zyq9vh65aGM", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [event.host] of type [keyword]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:77"}}}}}
I installed the ELK Stack fresh.
Filebeat is shipping logs from snort maschine to the server where the ELK stack is running. Further i see the Index in Index Management of Elasticsearch .
As far as i understand, there must be some kind of problem to correctly index the Data.
I am quite new to this topic and not aware of any possiblity to debug this any further
Thank you all in advance
Greetings
Simon
P.S.: Sry for posting in the wrong project.
I have same issue on logstash-plain.log
when I flow the below link
https://github.com/robcowart/synesis_lite_snort
2019-02-19T17:18:15,169][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snort-1.0.0-2019.02.19", :_type=>"logs", :_routing=>nil}, 2019-02-19T06:12:57.313Z %{host} %{message}], :response=>{"index"=>{"_index"=>"snort-1.0.0-2019.02.19", "_type"=>"logs", "_id"=>"t_NnBGkBJ7r60F19pLY4", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [event.host] of type [keyword]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:508"}}}}}
Same problem! =/
If you are using elastic 7.x the index will not work. You have to use 6.x or change the template.
I have no working template yet but using default mapping is not allowed anymore.
Loading the current template in 7.x will give you the Could not index event to Elasticsearch. {:status=>400, :action= error
https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html
"mappings": {
"default": {
"numeric_detection": true,
"dynamic_templates": [
{
"string_fields": {
"match_mapping_type": "string",
"match": "*",
"mapping": {
"type": "keyword"
}
}
}
],
Elasticsearch 7.x
Specifying types in requests is deprecated. For instance, indexing a document no longer requires a document type. The new index APIs are PUT {index}/_doc/{id} in case of explicit ids and POST {index}/_doc for auto-generated ids. Note that in 7.0, _doc is a permanent part of the path, and represents the endpoint name rather than the document type.
The include_type_name parameter in the index creation, index template, and mapping APIs will default to false. Setting the parameter at all will result in a deprecation warning.
**The _default_ mapping type is removed.**
Closing all issues as this project has been archived.