ClickHouse/clickhouse-cpp

how to read a column of LowCardinality(Nullable(string))

changvin opened this issue · 6 comments

version: 2.4.0
gcc version: 8.3.1
system: centos 7, linux version 3.10.0-957.el7.x86 64

Hi, I'm new to clickhouse so I'm testing it.
but I facing a type cast problem.
this my code :
1694159148859

but it seem no right, print std::bad_cast
1694158862679

Tasks

No tasks being tracked yet.
Enmk commented

Hi @changvin ! Thanks to @den818 you can do the following:

auto col = column->As<ColumnLowCardinalityT<ColumnNullableT<ColumnString>>>();

Then col->At(0) is going to return std::optional<std::string>.

There is a test case for it (might be a bit convoluted) right here:

TEST_P(RoundtripCase, LowCardinalityTNullableString) {

Enmk commented

Btw, next time, please attach the code as text, not as a picture... it is much easier to play around and/or modify it like that.

///@Enmk
//> Hi @changvin ! Thanks to @den818 you can do the following:

auto col = column->As<ColumnLowCardinalityT<ColumnNullableT<ColumnString>>>();

Then col->At(0) is going to return std::optional<std::string>.

There is a test case for it (might be a bit convoluted) right here:

TEST_P(RoundtripCase, LowCardinalityTNullableString) {

this is a insert LowCardinalityTNullableString column test case not read ,

when read a LowCardinalityTNullableString column , col == nullptr is true

/// use by LowCardinalitv(Nullable(string))
void read_lowcardinality_nullable_str(clickhouse::ColumnRef column, std::vector<std::string>& output) {
	auto col = column->As<clickhouse::ColumnLowCardinalityT<clickhouse::ColumnNullableT<clickhouse::ColumnString>>>();
	if (col == nullptr) {
		std::cerr <<"read_lowcardinality_nullable_str col is null"<<std::endl;
		return;
	}
	for(size_t i = 0;  i < col->Size(); i++) {
		auto op = col->At(i);
		if (op.has_value())
			output.emplace_back(*op);
		else
			output.emplace_back("");
	}
}

the result is

read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
read_lowcardinality_nullable_str col is null
...
Enmk commented

Please check master now, most likely you've had ClientOptions::backward_compatibility_lowcardinality_as_wrapped_column set to true and that's why the LowCardinality(String) column was unwrapped into just String.

请现在检查大师,很可能您已经ClientOptions::backward_compatibility_lowcardinality_as_wrapped_column设置为true,这就是该LowCardinality(String)列已展开为String

@Enmk
Thank you for your reply,
I'm pretty sure ClientOptions::backward_compatibility_lowcardinality_as_wrapped_column set to false before,maybe ColumnLowCardinalityT not work with ColumnNullableT when read ?

Enmk commented

请现在检查大师,很可能您已经ClientOptions::backward_compatibility_lowcardinality_as_wrapped_column设置为true,这就是该LowCardinality(String)列已展开为String

@Enmk Thank you for your reply, I'm pretty sure ClientOptions::backward_compatibility_lowcardinality_as_wrapped_column set to false before,maybe ColumnLowCardinalityT not work with ColumnNullableT when read ?

Please see #339

This one confirms that it works

Perhaps, it would be helpful to know the real type of column at run-time with:

column->GetType().GetName()

you can include that in your error message for debugging.