SearchResponse has empty jsonStr for queries with sourceFilter

Hi there,

I am using the elastic search client and it is quite nice, so thanks for your work.

I have a query that uses sourceFilter and to my surprise in the SearchResponse jsonStr is empty.

I found the cause

elasticsearch-client/elasticsearch-core/src/main/scala/com/sumologic/elasticsearch/restlastic/RestlasticSearchClient.scala

Line 81 in e41b4b2

val jsonStr = if(query.sourceFilter.isDefined) "" else rawJson.jsonStr

I can workaround the issue using the rawSearchResponse.

I just would like to know, if there is a reason for that line to be there.

Best regards,
Jonas.

@CCheSumo may know. Seems like it could be a bug, we'll look into it. Thanks for the report!

Thanks

Hi @jonasanso,

Any reason your are not using the RawSearchResponse? It is the parsed out version of the raw jsonStr. We decided not to pass back the raw jsonStr when source filter is supplied (for performance benefits). Again, all the same information are available in RawSearchResponse. It is recommended to use it rather than raw jsonStr.

In the extreme scenario, if you really really want to use the raw jsonStr (with source filtering enabled), you can simply convert RawSearchResponse to a raw jsonStr on the local client side. There is no need to pass the same duplicate content over the internet twice only because they are in different formats.

Hi @CCheSumo,

Thanks for your answer.

I am migrating our code base from a different ES client and I already have the parsing logic to our models implemented in spray-json.

So for me is more useful the jsonStr than the RawSearchResponse.

You are right I can get around it and formatting RawSearchResponse to string and parsing it back.

But the question is, why SearchResponse contain the same data in two different ways, string and parsed using json4s?

For my needs, I find it OK because I can usey parsers, other people might prefer the already parsed response. But in the current scenario where we have both values, I find it strange that the jsonStr is missing when filtering, in fact when filtering we are reducing the size of jsonStr.

I am totally with you that there is no need to have this two attributes in memory and for the clarity of the API there should be only one. But consider the impact of forcing one json library.

Again, thanks for your efforts,
Jonas

@jonasanso,

That is a very nice explanation on the use case. I propose to add an boolean option in the query API (default to Some(true)) to indicate whether raw jsonStr should be passed back. By default, the jsonStr will also be passed back. For performance critical missions, user specify only passing back only the RawSearchResponse.

For backward compatibility, we will always pass back RawSearchResponse now. At some point, with breaking version change, we can use the same boolean option to specify to use either parsed jsonStr or raw jsonStr.

How does it sound to you?

Regards,

Sounds great!

Thanks @CCheSumo