Alexander-Barth/NCDatasets.jl

unable to read nc file exported by xee

Closed this issue · 6 comments

kongdd commented 
using NCDatasets
f = "xee-test.nc"
NCDataset(f)

"xee-test.nc" is exported by xee, and available at: xee-test.zip

Details can be found at: google/Xee#171

Gives the error:

julia> NCDataset(f)
ERROR: MethodError: Cannot `convert` an object of type Vector{Float64} to an object of type String

Closest candidates are:
  convert(::Type{String}, ::Base.JuliaSyntax.Kind)
   @ Base C:\workdir\base\JuliaSyntax\src\kinds.jl:975
  convert(::Type{String}, ::String)
   @ Base essentials.jl:321
  convert(::Type{String}, ::FilePathsBase.AbstractPath)
   @ FilePathsBase C:\Users\hydro\.julia\packages\FilePathsBase\Laclt\src\path.jl:117
  ...

Stacktrace:
 [1] setindex!(h::Dict{String, String}, v0::String, key0::Vector{Float64})
   @ Base .\dict.jl:367
 [2] initboundsmap!(ds::NCDataset{Nothing, Missing})
   @ CommonDataModel C:\Users\hydro\.julia\packages\CommonDataModel\G3moc\src\dataset.jl:284
 [3] NCDataset(ncid::Int32, iswritable::Bool, isdefmode::Base.RefValue{Bool}; parentdataset::Nothing, maskingvalue::Missing)
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:89
 [4] NCDataset
   @ C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:65 [inlined]
 [5] NCDataset(filename::String, mode::String; format::Symbol, share::Bool, diskless::Bool, persist::Bool, memory::Nothing, maskingvalue::Missing, attrib::Vector{…})
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:235
 [6] NCDataset
   @ C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:170 [inlined]
 [7] NCDataset(filename::String)
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:170
 [8] top-level scope
   @ c:\Users\hydro\github\CUG-hydro\xee_abc\test-NcData.jl:11

While, it is normal with NetCDF.jl and R terra.

using NetCDF
f = "xee-test.nc"
lon = ncread(f, "lon")
lat = ncread(f, "lat")
A = ncread(f, "surface_net_solar_radiation")

NCDatasets: v0.14.5

Alexander-Barth commented

Thank you for your reproducible bug report.
In fact, according to the CF convention the bounds attribute is expected to be a string (a containing a variable name with related metadata):
https://cfconventions.org/Data/cf-conventions/cf-conventions-1.11/cf-conventions.html#cell-boundaries

This is what I see with the tool ncdump:

netcdf xee-test {
dimensions:
        time = 10 ;
        lon = 78 ;
        lat = 48 ;
variables:
        float surface_net_solar_radiation(time, lon, lat) ;
                surface_net_solar_radiation:_FillValue = NaNf ;
                surface_net_solar_radiation:id = "surface_net_solar_radiation" ;
                surface_net_solar_radiation:data_type = "{\'type\': \'PixelType\', \'precision\': \'double\'}" ;
                surface_net_solar_radiation:dimensions = 3601LL, 1801LL ;
                surface_net_solar_radiation:crs = "EPSG:4326" ;
                surface_net_solar_radiation:crs_transform = 0.1, 0., -180.05, 0., -0.1, 90.05 ;
                surface_net_solar_radiation:scale_factor = 0.1 ;
                surface_net_solar_radiation:bounds = 108.5, 28.75, 116.25, 33.5603826424351 ;
[...]

xee seem to reuse the attribute bounds for something else (probably the bounding box).
I will update NCDatasets/CommonDataModel to ignore the bounds attribute if it has not an expected type.

Alexander-Barth commented

Also a coordinate variable must not have the _FillValue or missing_value attributes:

https://github.com/cf-convention/cf-conventions/blob/main/conformance.adoc#5-coordinate-systems-and-domain

        double lon(lon) ;
                lon:_FillValue = NaN ;
        double lat(lat) ;
                lat:_FillValue = NaN ;
kongdd commented

Thanks for your quick response. I think ignore the wrong format bounds attribute is a acceptable option.

kongdd commented

Also a coordinate variable must not have the _FillValue or missing_value attributes:

https://github.com/cf-convention/cf-conventions/blob/main/conformance.adoc#5-coordinate-systems-and-domain

        double lon(lon) ;
                lon:_FillValue = NaN ;
        double lat(lat) ;
                lat:_FillValue = NaN ;

Thank you. Maybe I need to post those information to xee developer.

  • 👍1
Alexander-Barth commented

Maybe I need to post those information to xee developer.

Yes, that would be great and reduce some troubles with other software.