[question] Struggling with update to 0.14

[jasal82]

Hi,
I'm currently trying to migrate my existing project to 0.14 so that I can use the new macros, but I'm struggling with my ZipWriter implementation. KotoObject now implies KotoSync and that seems to break some things. Here's my Zip struct:

#[derive(Clone, KotoCopy, KotoType)]
struct Zip {
    inner: PtrMut<ZipWriter<File>>,
    needs_finish: bool,
}

This gives me the following error when I try to create a koto_impl for it:

error[E0277]: `NonNull<zstd_sys::ZSTD_CCtx_s>` cannot be shared between threads safely
   --> src\scripting\api\arch.rs:181:21
    |
181 | impl KotoObject for Zip {
    |                     ^^^ `NonNull<zstd_sys::ZSTD_CCtx_s>` cannot be shared between threads safely
    |
    = help: within `ZipWriter<std::fs::File>`, the trait `KotoSync` is not implemented for `NonNull<zstd_sys::ZSTD_CCtx_s>`, which is required by `api::arch::Zip: KotoSync`
note: required because it appears within the type `zstd_safe::CCtx<'static>`
   --> C:\Users\asaljo\.cargo\registry\src\index.crates.io-6f17d22bba15001f\zstd-safe-5.0.2+zstd.1.5.2\src\lib.rs:156:12
    |
156 | pub struct CCtx<'a>(NonNull<zstd_sys::ZSTD_CCtx>, PhantomData<&'a ()>);
    |            ^^^^
note: required because it appears within the type `zstd::stream::raw::Encoder<'static>`

My program is single-threaded anyway so I thought maybe switching the underlying PtrMut implementation from Arc to Rc would help but I didn't manage to use the rc feature instead of arc (always getting the double-feature warning from koto_memory, although I disabled default-features).

Any help would be appreciated.

[irh]

Hi @jasal82, I just double-checked that using non-Send types works with Koto's rc feature in this test repo. The setup in the test app is very simple, so I guess there's something else going on with the way your project's making use of Koto? Maybe the arc feature is creeping in to the picture somehow? Do you have a branch you can point me to that demonstrates the problem?

[jasal82]

Yes, as I said I tried exactly this procedure (disabling default features and using rc, in koto and all extension modules) but I always end up with rc and arc features during compilation somehow. I'll try to debug the build some more today.

[jasal82]

Even with these dependencies (note explicit koto_memory reference)

koto = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_json = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_memory = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_regex = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_serialize = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_tempfile = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_toml = { version = "0.14.0", default-features = false, features = ["rc"] }
koto_yaml = { version = "0.14.0", default-features = false, features = ["rc"] }

I get redefinition errors from koto_memory:

error[E0428]: the name `make_ptr` is defined multiple times
  --> C:\Users\asaljo\.cargo\registry\src\index.crates.io-6f17d22bba15001f\koto_memory-0.14.0\src\rc\ptr.rs:18:1
   |
18 | macro_rules! make_ptr {
   | ^^^^^^^^^^^^^^^^^^^^^ `make_ptr` redefined here
   |
  ::: C:\Users\asaljo\.cargo\registry\src\index.crates.io-6f17d22bba15001f\koto_memory-0.14.0\src\arc\ptr.rs:18:1
   |
18 | macro_rules! make_ptr {
   | --------------------- previous definition of the macro `make_ptr` here
   |
   = note: `make_ptr` must be defined only once in the macro namespace of this module

and this error:

error: A single memory management feature can be enabled at a time
  --> C:\Users\asaljo\.cargo\registry\src\index.crates.io-6f17d22bba15001f\koto_memory-0.14.0\src\lib.rs:16:1
   |
16 | compile_error!("A single memory management feature can be enabled at a time");
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[irh]

Ok I see the problem now, the libs that depend on koto_serialize aren't disabling default features, which is how the arc feature creeps in.

I'm going to work on a quick fix now and then play around a bit to see if there's a cleaner approach to selecting the rc build type, I don't like how cumbersome this is.

[jasal82]

Adding a koto_json dependency in your test project is already enough to reproduce the error. It seems that feature transitivity is not working correctly on the modules.

[irh]

Yes, exactly. It's because koto_json depends on koto_serialize without disabling default features, which causes arc to be enabled.

[jasal82]

I think it's the koto_serialize dependency:

├── koto_json feature "rc"
│   ├── koto_json v0.14.0
│   │   ├── koto_runtime v0.14.0 (*)
│   │   ├── koto_serialize feature "default"
│   │   │   ├── koto_serialize v0.14.0
│   │   │   │   ├── koto_runtime v0.14.0 (*)
│   │   │   │   └── serde feature "default"
│   │   │   │       ├── serde v1.0.203
│   │   │   │       └── serde feature "std"
│   │   │   │           └── serde v1.0.203
│   │   │   └── koto_serialize feature "arc"
│   │   │       ├── koto_serialize v0.14.0 (*)
│   │   │       └── koto_runtime feature "arc"
│   │   │           ├── koto_runtime v0.14.0 (*)
│   │   │           └── koto_memory feature "arc"
│   │   │               ├── koto_memory v0.14.0 (*)
│   │   │               └── koto_memory feature "parking_lot"
│   │   │                   └── koto_memory v0.14.0 (*)

[jasal82]

Sorry, I just posted stuff without getting your updates in the thread

[irh]

Lol I did wonder if my replies were getting through 😆

There's a quick fix on #342, and I think this problem is annoying enough to put in to a 0.14.1 release, but I'll investigate if there's a better general approach first.

[jasal82]

Sounds reasonable. It's a bit annoying having to set the feature on all of the dependencies. Thanks for the quick response.

[irh]

I think there's a cleaner approach (only exposing arc/rc features in leaf-node crates like koto or koto_runtime), but removing the existing rc/arc features would be a breaking change so can't go in to a 0.14.1 release.

I'll backport the quick fix, and then the cleanup changes will go onto main.

[irh]

I've just published 0.14.1 which includes the fix for this issue.