Inexact error when saving large data without compresssion

Question

Inexact error when saving large data without compresssion

oluwandabira opened this issue a year ago · 1 comments

Originally this issue on Metalhead, I'm opening an issue here because it appears to be a JLD2 issue. It is not the same as this issue as I'm not using compression so the problem is from JLD2 not the compression library.

Saving Metalhead model with JDL2 fails if the model is large enough (at least I think that's what's happening)

using Flux
using Metalhead
using JLD2

model = ResNet(152)
model_state = Flux.state(model)
jldsave("model_state.jld2"; model_state)

The above will fails with:

InexactError(:trunc, UInt16, 66886)

Full Stack trace

Also fails for EfficientNet(:b8)

Pkg.status():

[587475ba] Flux v0.14.6
[033835bb] JLD2 v0.4.35
[dbeba491] Metalhead v0.8.3

Answer 1 · 2023-10-07T15:21:55.000Z

Hi @oluwandabira,

you guessed correctly.
This is as problem of size.
Flux / Metalhead seems to encode the whole structure of the model in a type stable manner within a NamedTuple.
(type signature below)
JLD2 / HDF5 limit their type descriptions to <64kb. (This is quite reasonable for most cases..)
I don't think there is an easy fix on the JLD2 side.

For reference: if I remove the type info by recursively wrapping tuples and namedtuples, it works just fine:

julia> struct UntypedNamedTuple{N}
           names::NTuple{N,Symbol}
           fields::Any
       end

julia> function recursively_convert_to_untyped(nt::NamedTuple)
           names = (keys(nt)...,)
           fields = (recursively_convert_to_untyped.(values(nt))...,)
           return UntypedNamedTuple(names, fields)
       end
recursively_convert_to_untyped (generic function with 3 methods)

julia> function recursively_convert_to_untyped(nt::Tuple)
           fields = (recursively_convert_to_untyped.(values(nt))...,)
           return Ref{Any}(fields)
       end
recursively_convert_to_untyped (generic function with 3 methods)

julia> recursively_convert_to_untyped(o) = o
recursively_convert_to_untyped (generic function with 3 methods)

julia> data = recursively_convert_to_untyped(model_state);

julia> jldsave("test.jld2"; data)

julia> dump(typeof(model_state))
NamedTuple{(:layers,), Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, Vararg{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, 7}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, Vararg{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, 35}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}, NamedTuple{(:connection, :layers), Tuple{Tuple{}, Tuple{Tuple{}, NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}, Tuple{}, NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{Tuple{}, Array{Float32, 4}, Bool, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}}, NamedTuple{(:λ, :β, :γ, :μ, :σ², :ϵ, :momentum, :affine, :track_stats, :active, :chs), Tuple{Tuple{}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Vector{Float32}, Float32, Float32, Bool, Bool, Nothing, Int64}}}}}}}}}}}}}}, NamedTuple{(:layers,), Tuple{Tuple{Tuple{}, Tuple{}, NamedTuple{(:weight, :bias, :σ), Tuple{Matrix{Float32}, Vector{Float32}, Tuple{}}}}}}}}}}} <: Any