Occasional Failed to derive xcontent in bulk index

[bejayoharen]

I am bulk indexing content, and decided to overwrite the sender function so that:

1. I could retry failures
2. I could get a report of errors

I'm indexing a small amount of content (My service reports about 500 writes/day) and every so often (maybe every two days or so), I get this error message:

{"error":{"root_cause":[{"type":"parse_exception","reason":"Failed to derive xcontent"}],"type":"parse_exception","reason":"Failed to derive xcontent"},"status":400}

From what I understand of elastic search, this means the data is badly formatted, so I printed out the buffer that's being sent to the bulk method and it appears to contain a single colon character.

I'm scratching my head to try to figure out what input I could have given the library to make it try to send invalid data to elastic search. I don't think I've done anything unusual, just using the standard BulkIndex functions that elastigo provides.

I haven't seen this in my test environment, but I'll do additional tests in case I missed it.

I'm sorry I can't provide much more info that this at this time, but I thought I would report it in case you have suggests for further debugging I can do or in case anyone else is seeing this. I realize this bug report might be useless at this point :(

Here is my sender method:

bulkIndexer.Sender = func(buf *bytes.Buffer) error {
		var respJson []byte
		var err error = nil
		tries := 15
		for try := 0; try < tries; try++ {
			// sleeps up to about 3 minutes:
			time.Sleep(time.Duration(try*try) * time.Second)
			// do it:
			respJson, err = c.DoCommand("POST", "/_bulk", nil, buf)
			if err == nil {
				break
			}
			if try < tries-1 {
				WARN.Println("Error in bulk indexing:", err, "Will retry.")
			}
		}
		if err != nil {
			message := "There was an error in bulk indexing after " + strconv.Itoa(tries) + " tries\n"
			message += "Error: " + string(respJson) + "\n"
			message += "Bulk Message: " + buf.String()
			ERROR.Println(message)
			EmailAdminErrorRateLimited("Error Indexing search", message, err, "searchtasks")
		}
		return err
	}
	bulkIndexer.Start()
})