coreylowman/dfdx

Replace explicit features and paths on generated code

swfsql opened this issue · 0 comments

swfsql commented

Some generated code has explicit feature detection and paths, such as #[cfg(feature = "safetensors")] and ::safetensors::Dtype, but those would require the user app code (which depends on the dfdx lib) to have that feature, and to also directly depend on the safetensors lib.

An example:

quote! {
#[cfg(feature = "safetensors")]
impl #built_impl dfdx_core::nn_traits::SaveSafeTensors for #builder_name #built_ty #built_where {
fn write_safetensors(
&self,
location: &str,
tensors: &mut Vec<(String, ::safetensors::Dtype, Vec<usize>, Vec<u8>)>,
) {}
}

For the feature detection, ideally it happens during code generation (but not be present in the generated code).
For the safetensors dependency, it could be best to re-export it from the dfdx and then refer to it in the generated code. This problem also exists for when referencing dfdx_core and dfdx_derives.

Regarding the later, I'm not sure what's the best way forward. There are libs such as proc-macro-crate that may help for the crate names that gets generated, but I believe there are other options such as allowing user-defined names like how serde does.