Big opaque terms
Closed this issue · 1 comments
zgbjgg commented
Check if can change opaque term representing dataframes or series to a single key in the python process, so dataframe will live only in python side.
Running large datasets maybe will cause invalid_protocol_data
but I'm not sure about, however this will be a great improve.
zgbjgg commented
The error invalid_protocol_data is thrown when using the same worker for different process, for example, sending a dataframe to get max and then sending other dataframe to get min, since the port on erlang side will colapse decoding data, however this will be avoid with a mechanism of pooling such as poolboy, checkout and checkin the worker 👍