c42f/RemoteREPL.jl

Use a serialization library that is bidirectional between julia versions

Opened this issue · 1 comments

It would be nice to move away from the standard library Serialization, as it only guarantees that newer julia versions can read files written by older ones, and not the other way. Bi-directionality is needed.

Based on the recommendations in https://fluxml.ai/Flux.jl/stable/saving/, options are https://github.com/JuliaIO/JLD2.jl and https://github.com/JuliaIO/BSON.jl. Perhaps lightweightness should be a priority. On my system, in a temp environment, I get

julia> @time using JLD2
  0.312673 seconds (316.65 k allocations: 18.135 MiB)

julia> @time using BSON
  0.002721 seconds (3.12 k allocations: 288.133 KiB)

Also, BSON.jl has no dependencies, so I guess there is less that can go wrong there? It seems appropriate to use a minimal low-dependency library for serialization in the context of RemoteREPL, so I suggest BSON.jl. Is this a direction that can be agreed on?

c42f commented

Yes we can do this and it's very desirable to have independence on the Julia version.

It may be tricky to support all features, however - can either of these serializations support arbitrary Julia objects? @remote relies on Serialization for transporting arbitrary Julia objects.