Incorrect type for nested fields

[dmarkhas]

In our index we have a field called "system" that has two nested properties - host (String) and sampletime (Date).
Using WorkbenchJ, the following query works OK:

SELECT system.host, system.sampletime
FROM "index"."data-generator-poc-async"."arm" limit 1

But the following queries fail with an error "Value in column '82' is not a Date but is String" (column 82 is system.sampletime):

SELECT system
FROM "index"."data-generator-poc-async"."arm" limit 1

SELECT *
FROM "index"."data-generator-poc-async"."arm" limit 1

[corneversloot]

For some reason we have a lot of these typing issues now which I did not encounter before. What does WorkbenchJ show when you use the 'Database Explorer' and select the 'arm' table? Doing this should show all the columns and their types and im interested what it thinks about your nested types.

ps. why do you prefix it with "index"."data-generator-poc-async" that should not be needed when you are connected with the "data-generator-poc-async" index right?

[dmarkhas]

Database Explorer shows the following:

system  java.lang.Object    NO  NO      NO  NO  {properties={host={type=string, fields={raw={type=string, index=not_analyzed, ignore_above=10000}}}, sampletime={type=date, format=strict_date_optional_time||epoch_millis}}}   2000    0

"system.host"   string(1)   NO  NO      NO  NO  {type=string, fields={raw={type=string, index=not_analyzed, ignore_above=10000}}}   12  0

"system.sampletime" date    NO  NO      NO  NO  {type=date, format=strict_date_optional_time||epoch_millis} 91  0

Squirrel succeeds in running the same query semi-successfully - if I do "select system" then I get two columns in the result (system.sampletime and system.host) but system.sampletime only has "Error" on each line. If I do "select system.sampletime, system.host" then both columns are populated correctly.

p.s. the prefix is indeed redundant, it was there because I copied the statement from a Drill query, but it doesn't change the behavior - running the query without the "index" prefix results in the same issue.

[corneversloot]

I did some digging using an old dataset with lots of nesting and it seems this issue has to do with the Date type and i guess it is related with the other issue as well. Requesting nested objects that do not have a Date works fine but if there is a date somewhere it breaks. Not sure why it happens though and might take some time to figure it out

[corneversloot]

Hi Dan,
I have implemented a fix for this issue which now works on my testset with multiple nested objects. Please use the latest 0.8.2.4 release to test it on your own data and let me know if it works.

[dmarkhas]

I think it works, in Squirrel I am able to get both nested fields correctly when doing select system (where system is a compound type), however now there is strange behavior which I think may be related to #22 where in Workbench I don't get any results when running select * from :

select count(*) from arm - returns correct number of 2014567 documents
select * from arm - returns 0 rows

When using Drill, the same query returns exactly 10 rows, even though no such limit is specified..

[dmarkhas]

I commented in the other thread about the issue, and closing this one since this is now resolved.