r-barnes/ArcRasterRescue

4 bit data warps raster file

feefladder opened this issue · 3 comments

feefladder commented

I have a classified raster with 4 bit data stored in a geodatabase. When converting with ARR, the output was a weird striped output. I have included the .gdb, database with a single raster. The converted file with ArcRasterRescue is lu_arr.tif, and the rest is the converted file using ArcGIS.
lu.zip
The (verbose) output is given below:


/ArcRasterRescue/build/arc_raster_rescue.exe ./lu.gdb/ 0 ./lu_arr.tif
Arc Raster Rescue (url=github.com/r-barnes/ArcRasterRescue, hash=482eb367ba86b43c, compiled=2021-05-06 20:37:05)
Opening BaseTable as './lu.gdb/a00000001.gdbtable'...
gdbtables found:
        GDB_SystemCatalog - a00000001.gdbtable
        GDB_DBTune - a00000002.gdbtable
        GDB_SpatialRefs - a00000003.gdbtable
        GDB_Items - a00000004.gdbtable
        GDB_ItemTypes - a00000005.gdbtable
        GDB_ItemRelationships - a00000006.gdbtable
        GDB_ItemRelationshipTypes - a00000007.gdbtable
        GDB_ReplicaLog - a00000008.gdbtable
        lu_prado_20m - a00000009.gdbtable
        fras_ras_lu_prado_20m - a0000000a.gdbtable
        fras_aux_lu_prado_20m - a0000000b.gdbtable
        fras_blk_lu_prado_20m - a0000000c.gdbtable
        fras_bnd_lu_prado_20m - a0000000d.gdbtable
        VAT_lu_prado_20m - a0000000e.gdbtable
Opening BaseTable as './lu.gdb/a0000000d.gdbtable'...
Opening RasterBase as './lu.gdb/a0000000d.gdbtable'
Field name (type=1): sequence_nbr
Skipping field 'sequence_nbr'
Field value: 1
Field name (type=1): raster_id
Skipping field 'raster_id'
Field value: 1
Field name (type=4): name
Field value: ''
Field name (type=1): band_flags
Skipping field 'band_flags'
Field value: 67340801
Field name (type=1): band_width
Field value: 380
Field name (type=1): band_height
Field value: 583
Field name (type=1): band_types
band_types = 0 4 20 0 00000000 00000100 00100000 00000000
Detected band data type =
Field name (type=1): block_width
Field value: 128
Field name (type=1): block_height
Field value: 128
Field name (type=3): block_origin_x
Field value: 614456
Field name (type=3): block_origin_y
Field value: 4.18933e+06
Field name (type=3): eminx
Field value: 614456
Field name (type=3): eminy
Field value: 4.17769e+06
Field name (type=3): emaxx
Field value: 622036
Field name (type=3): emaxy
Field value: 4.18933e+06
Field name (type=1): cdate
Field value: 1620332591
Field name (type=1): mdate
Field value: 1620332591
Field name (type=1): srid
Skipping field 'srid'
Field value: 2
Using geotransform (this is experimental): 614456.00000000093132257462 19.94736842105263008307 0.00000000000000000000 4189334.00000000372529029846 0.00000000000000000000 -19.96569468267581370924
Using internal datatype 'h'. Use 'c++filt -t' to decode.
Opening BaseTable as './lu.gdb/a00000009.gdbtable'...
field.raster.wkt: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
field.shape.wkt: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]
Opening BaseTable as './lu.gdb/a0000000d.gdbtable'...
Opening RasterBase as './lu.gdb/a0000000d.gdbtable'
Field name (type=1): sequence_nbr
Skipping field 'sequence_nbr'
Field value: 1
Field name (type=1): raster_id
Skipping field 'raster_id'
Field value: 1
Field name (type=4): name
Field value: ''
Field name (type=1): band_flags
Skipping field 'band_flags'
Field value: 67340801
Field name (type=1): band_width
Field value: 380
Field name (type=1): band_height
Field value: 583
Field name (type=1): band_types
band_types = 0 4 20 0 00000000 00000100 00100000 00000000
Detected band data type =
Field name (type=1): block_width
Field value: 128
Field name (type=1): block_height
Field value: 128
Field name (type=3): block_origin_x
Field value: 614456.00000000093132257462
Field name (type=3): block_origin_y
Field value: 4189334.00000000372529029846
Field name (type=3): eminx
Field value: 614456.00000000093132257462
Field name (type=3): eminy
Field value: 4177694.00000000372529029846
Field name (type=3): emaxx
Field value: 622036.00000000093132257462
Field name (type=3): emaxy
Field value: 4189334.00000000372529029846
Field name (type=1): cdate
Field value: 1620332591
Field name (type=1): mdate
Field value: 1620332591
Field name (type=1): srid
Skipping field 'srid'
Field value: 2
Using geotransform (this is experimental): 614456.00000000093132257462 19.94736842105263008307 0.00000000000000000000 4189334.00000000372529029846 0.00000000000000000000 -19.96569468267581370924
Opening BaseTable as './lu.gdb/a0000000a.gdbtable'...
Opening RasterProjection as './lu.gdb/a0000000a.gdbtable'
Opening BaseTable as './lu.gdb/a0000000c.gdbtable'...
Allocating 1x384x640 = 245760
Opening Raster Data as ./lu.gdb/a0000000c.gdbtable
Skipped points: 0
Projection: PROJCS["ED_1950_UTM_Zone_30N",GEOGCS["GCS_European_1950",DATUM["D_European_1950",SPHEROID["International_1924",6378388.0,297.0]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",500000.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-3.0],PARAMETER["Scale_Factor",0.9996],PARAMETER["Latitude_Of_Origin",0.0],UNIT["Meter",1.0]]

I don't think it is related to the issue, but I also had some overflow warnings when building the program related to the nodata value.

```

Scanning dependencies of target arc_raster_rescue
make[2]: Warning: File 'CMakeFiles/arc_raster_rescue.dir/depend.make' has modification time 0.71 s in the future
[ 25%] Building CXX object CMakeFiles/arc_raster_rescue.dir/src/arr.cpp.o
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp: In instantiation of ‘RasterData::RasterData(std::string, const RasterBase&) [with T = unsigned char; std::string = std::__cxx11::basic_string]’:
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1246:20: required from ‘void ExportTypedRasterToGeoTIFF(std::string, std::string, int, std::string) [with T = unsigned char; std::string = std::__cxx11::basic_string]’
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1278:84: required from here
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:958:36: warning: unsigned conversion from ‘int’ to ‘unsigned char’ changes value from ‘-9999’ to ‘241’ [-Woverflow]
958 | resize(maxpx-minpx, maxpy-minpy, -9999);
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1132:13: warning: unsigned conversion from ‘int’ to ‘unsigned char’ changes value from ‘-9999’ to ‘241’ [-Woverflow]
1132 | no_data = -9999; //TODO: This cannot always be NoData.
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp: In instantiation of ‘RasterData::RasterData(std::string, const RasterBase&) [with T = signed char; std::string = std::__cxx11::basic_string]’:
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1246:20: required from ‘void ExportTypedRasterToGeoTIFF(std::string, std::string, int, std::string) [with T = signed char; std::string = std::__cxx11::basic_string]’
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1284:83: required from here
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:958:36: warning: overflow in conversion from ‘int’ to ‘signed char’ changes value from ‘-9999’ to ‘-15’ [-Woverflow]
958 | resize(maxpx-minpx, maxpy-minpy, -9999);
| ^~~~~
/mnt/c/Users/Joep/Documents/ArcRasterRescue/src/arr.cpp:1132:13: warning: overflow in conversion from ‘int’ to ‘signed char’ changes value from ‘-9999’ to ‘-15’ [-Woverflow]
1132 | no_data = -9999; //TODO: This cannot always be NoData.
| ^~~~~
[ 50%] Linking CXX static library libarc_raster_rescue.a
make[2]: warning: Clock skew detected. Your build may be incomplete.
[ 50%] Built target arc_raster_rescue
Scanning dependencies of target arc_raster_rescue.exe
make[2]: Warning: File 'CMakeFiles/arc_raster_rescue.exe.dir/depend.make' has modification time 0.7 s in the future
[ 75%] Building CXX object CMakeFiles/arc_raster_rescue.exe.dir/evaluation/main.cpp.o
[100%] Linking CXX executable arc_raster_rescue.exe
make[2]: warning: Clock skew detected. Your build may be incomplete.
[100%] Built target arc_raster_rescue.exe

</details>

I hope this helps!
r-barnes commented

I've moved the no data issue to #18: please try to keep issues separate where possible.

I think somewhere I have an 8-bit assumption hard-coded. If you had the exact same dataset saved to, e.g., 4-bit, 8-bit, 16-bit, and 32-bit integers it might be possible to figure out how Arc is indicating a 4-bit encoding.

feefladder commented

I'll see if I can get that data! Would it be four .gdb files with those datasets or one with four datasets?

As for splitting the issues, I was not sure if it was rrlated, so I didn't want to make an issue just yet. Thanks for making it!

feefladder commented

So I've added the .gdb files:
LUs.zip
it's a bit funny though: In the properties tab, the data is given as char for 8-bit. (short for 16 bit and signed int for 32 bit are normal I think.)