morganstanley/binlog

not support __int128 output

Opened this issue · 6 comments

Question1:
template <typename T, int32_t Digits> class TestFixedPoint {
public:
TestFixedPoint(T t) : value_(t) {}

private:
T value_;
BINLOG_ADAPT_STRUCT_FRIEND;
};

using TestPrice = TestFixedPoint<int64_t, 9>;
BINLOG_ADAPT_TEMPLATE((typename Interl, int Digits),
(TestFixedPoint<Interl, Digits>), value_)
BINLOG_INFO_W(writer, "my log MyPrice: {}", myPrice);
Compile time error:
binlog/include/mserialize/make_template_serializable.hpp:75:84: 错误:‘long int TestFixedPoint<long int, 9>::value_’ is private within this context
Question2:
if T == __int128 then throw "This Arithmetic type is not taggable" at tag.hpp line 79

I'm unable to reproduce the first issue:

template <typename T, int32_t Digits>
class TestFixedPoint {
public:
  TestFixedPoint(T t) : value_(t) {}

private:
  T value_;
  BINLOG_ADAPT_STRUCT_FRIEND;
};

using TestPrice = TestFixedPoint<int64_t, 9>;
BINLOG_ADAPT_TEMPLATE((typename Interl, int Digits), (TestFixedPoint<Interl, Digits>), value_)

//...

  TestPrice myPrice(1);
  BINLOG_INFO("my log MyPrice: {}", myPrice);

This works as expected.

Regarding __int128: It is not a standard type, and therefore not supported. As a workaround, you can:

  • Add two getters, that return the upper and lower halfs, and log them.
  • Add a getter that copies the value into a 16 byte array, and log that
  • Add a getter, that converts the value to a string, and log that

Hi Erenon,
Thanks for your reply.

The origin requirement is that a custom type (similar to java.math.BigDecimal) is introduced represent arbitrary-precision signed decimal numbers, definition is as following, integerT = __int128

template<typename IntegerT, int32_t Digits>
class FixedPoint {
// custom defined operator + - * /

std::toString() const {
 ...
}

private:
IntegerT _value;
};

using ValueT = FixedPoint<__int128, 13>;

ValueT::toString() involves converting __int128 to string, which is time-consuming, binlog is a good candidate to solve this problem.

we can write FixedPoint as 3 elements per your 1st suggestion.

  1. Add two getters, that return the upper and lower halfs, and log them.
  2. Digits
template<typename IntegerT, int32_t Digits>
class FixedPoint {
// custom defined operator + - * /

std::toString() const {
 ...
}

int64_t getUper() const ;
int64_t getLower() const;
int32_t getDigits() const;

private:
IntegerT _value;
};

If we changed the code as above, binlog bread tool would take FixedPoint as normal json instead of the expected output fromFixedPoint::toString

so the question is how to customize the output of a struct? how to change bread tool code.

You can modify PrettyPrinter here: https://github.com/morganstanley/binlog/blob/main/include/binlog/PrettyPrinter.cpp#L128
to make it recognize your custom type (if (sb.name == "FixedPoint")) and format the output accordingly. There are already a few example there, that should help you get started.

Hi Erenon,
Thanks for your advice, we managed to print out FixedPoint<int128_t>.
now we 're about to turnover to prod, but found the output message is cluttered with types, things can go worse if FixedPoint<int128_t> is used in a struct.

for example ,
bread give output _orderQtyEq: engine::datatype::FixedPoint{ getBinlogValue: 0.003 }
expected is _orderQtyEq: 0.003

how to remove the type prefix , only keep the value?

erenon commented

did you change PrettyPrinter.cpp?

Erenon, thanks for your support.

After binlog turnedover to prod, it reduced CPU usage to one-half and reduced the CPU spikes with ~60M memory usage increase.