`Half` type arrays

[martindevans]

Problem

It isn't possible to create a numpy array from a Half[]

Example

This code:

var x = new[] { (Half)1, (Half)2, (Half)3 };
var y = np.array(x);

Produces this error:

System.ArgumentException: 'Can not convert type of given object to dtype: System.Half[]'

Expected

I expected this to create a new array with dtype == float16

[henon]

This is the first time I learn of the new Half type. I can support it of course

[henon]

Hmm, that is actually not that easy. Half is only available in .Net 5 and Numpy is still NetStandard 2.0

[henon]

Here is a workaround for you. You need to do some bit fiddling and you need to know exactly how float16 numbers are represented in binary. Then you can create a byte[] with two bytes per Half and create a numpy array from that like this:

        [TestMethod]
        public async Task F16Workaround()
        {
            // use byte array to create float16 array

            // numbers in float16:
            // 0 01111 0000000000 = 2^0 * (1 + 0/1024) = 1 
            // 1 01111 0000000000 = -1 * 2 ^ 0 * (1 + 0 / 1024) = -1
            // 0 11110 1111111111 = -1^(0) * 2^(15) * (1 + 1023/1024) ≈ 65504
            // see: https://devblogs.microsoft.com/dotnet/introducing-the-half-type/

            // these bytes in binary notation correspond to f16 numbers 1, -1 and 65504 (float16 max value)
            // note, the bytes are in reversed order to the bits shown above
            var bytes = new byte[] { 
                0b00000000, 0b00111100, // 1
                0b00000000, 0b10111100, // -1
                0b11111111, 0b01111011, // 65504
            };
            var floats = np.zeros(new Shape(3), np.float16);
            Console.WriteLine(floats.repr);
            // note, the using prevents a mem-leak with ctypes
            using (var ctypes = floats.PyObject.ctypes) { 
                long ptr = ctypes.data;
                Marshal.Copy(bytes, 0, new IntPtr(ptr), bytes.Length);
            }
            Console.WriteLine(floats.repr);
            Assert.AreEqual("array([ 1.00e+00, -1.00e+00,  6.55e+04], dtype=float16)", floats.repr);
        }

I guess you can probably directly copy the bytes from a Half[] into a byte[] with Marshal.Copy or so.

[martindevans]

Thanks, I'll give that a go tomorrow.

[henon]

I'll also see if I can get multi targeting to work so I can support Half natively but it might take some time

[henon]

@martindevans Sorry, I didn't have time to look into multi-targeting yet. Did the workaround help you?

[martindevans]

I did test it a couple of weeks ago but in the end I decided to stick with float32, just for simplicity.