Convert OpenCvSharp.Mat to Sharp.Mat and vice versa ?
minhduc66532 opened this issue · 10 comments
The reason why i want to do this is because i like the "completeness" of OpenCvSharp compare to SharpCv and fantantic syntax/flexible of SharpCv when working with NDArray and Tensor. So i want to use OpenCvSharp to do all of the image processing stuff => convert OpenCvSharp.Mat to Sharp.Mat => working with NDArray/Tensor stuff. Any ideas ?
I think you need to write a adapter between the two different data structure.
Another approach is help us to add missing APIs for SharpCV.
Ohh that sucks, well i really want to help but my knowledge about writing C++ wrapper in C# is VERY limited since i haven't done such thing before. But thank for your answer
Ok so u can use this SharpCV.Mat mat = new SharpCV.Mat(OpenCvSharp.Mat.CvPtr)
Hi @minhduc66532 , I'm in the same situation as you (I'm missing imencode and imdecode). Thanks for posting your solution for SharpCV => OpenCvSharp, did you work out how to convert back the other way too?
Edit:
Actually, I found a way:
image = Cv2.ImDecode(imageBytes, ImreadModes.AnyColor);
var outImage = new SharpCV.Mat(image.Clone().CvPtr);
...
using (var saveImage = image.Clone()) // Clone the original image so the Mat is the right size
{
outImage.data.CopyTo(saveImage.Data);
Cv2.ImEncode(".jpg", saveImage, out var bytes);
}
PS, thanks to all authors of this library. It has made translating Python code using numpy and openCV so much easier!
Sorry for the late answer, i was sleeping at the time u posted the question (different time zone lol). Anyway
Thanks for posting your solution for SharpCV => OpenCvSharp, did you work out how to convert back the other way too?
I think you misunderstood my answer or mistyped some stuff my guy :D. Since my answer above is convert OpenCVSharp Mat to SharpCV Mat, NOT vice versa. The reason why i want to convert OCS Mat to SC Mat is NDArray. I'm doing some AI stuff with Tensorflow.NET so i need to convert from OCS Mat to NDArray. That why i need to convert between the two since SharpCV Mat has the GetData method and SharpCV do have those brain raping numpy slicing stuff too. Below is the code i'm using to DIRECTLY convert OCS Mat to NDArray (i did jonk most of the code from SharpCV Mat.GetData source code 😉)
public static unsafe NDArray OpenCvSharpToNDArray(Mat input, int channels = 3)
{
Shape shape = new Shape(new int[] { input.Height, input.Width, channels });
var i = IntPtr.Zero;
if (input.Type() == MatType.CV_32SC1 || input.Type() == MatType.CV_32SC2)
{
int* ptr = (int*)input.DataPointer;
var block = new UnmanagedMemoryBlock<int>(ptr, shape.Size, () => DoNothing(i));
var storage = new UnmanagedStorage(new ArraySlice<int>(block), shape);
return new NDArray(storage);
}
else if (input.Type() == MatType.CV_32FC1)
{
float* ptr = (float*)input.DataPointer;
var block = new UnmanagedMemoryBlock<float>(ptr, shape.Size, () => DoNothing(i));
var storage = new UnmanagedStorage(new ArraySlice<float>(block), shape);
return new NDArray(storage);
}
else if (input.Type() == MatType.CV_8UC1 || input.Type() == MatType.CV_8UC3)
{
byte* ptr = (byte*)input.DataPointer;
var block = new UnmanagedMemoryBlock<byte>(ptr, shape.Size, () => DoNothing(i));
var storage = new UnmanagedStorage(new ArraySlice<byte>(block), shape);
return new NDArray(storage);
}
else
throw new NotImplementedException($"Can't find type: {input.Type()}");
}
[MethodImpl(MethodImplOptions.NoOptimization)]
private static void DoNothing(IntPtr ptr)
{
var p = ptr;
}
Actually, I found a way:
Very nice, i'm gonna save it since i probably i'm gonna need it in the future :D.
You need to consider the number of channels to new Shape. And I found another way to do it without touching the 'unsafe' code.
Mat to NDArray
public static NDArray ToNDArray(this Mat mat)
{
var matType = mat.Type();
var channels = mat.Channels();
var size = mat.Rows * mat.Cols * channels;
var shape = channels == 1 ? new Shape(mat.Rows, mat.Cols) : new Shape(mat.Rows, mat.Cols, channels);
if (matType == MatType.CV_32SC1 || matType == MatType.CV_32SC2)
{
var managedArray = new int[size];
Marshal.Copy(mat.Data, managedArray, 0, size);
var aslice = ArraySlice.FromArray(managedArray);
return new NDArray(aslice, shape);
}
if (matType == MatType.CV_32FC1)
{
var managedArray = new float[size];
Marshal.Copy(mat.Data, managedArray, 0, size);
var aslice = ArraySlice.FromArray(managedArray);
return new NDArray(aslice, shape);
}
if (matType == MatType.CV_64FC1)
{
var managedArray = new double[size];
Marshal.Copy(mat.Data, managedArray, 0, size);
var aslice = ArraySlice.FromArray(managedArray);
return new NDArray(aslice, shape);
}
if (matType == MatType.CV_8UC1 || matType == MatType.CV_8UC3 || matType == MatType.CV_8UC4)
{
var managedArray = new byte[size];
Marshal.Copy(mat.Data, managedArray, 0, size);
var aslice = ArraySlice.FromArray(managedArray);
return new NDArray(aslice, shape);
}
throw new Exception($"mat data type = {matType} is not supported");
}NDArray to Mat
public static Mat ToMat(this NDArray nDArray) =>
new Mat(nDArray.shape[0], nDArray.shape[1], nDArray.GetMatType(), (Array)nDArray);
public unsafe static Mat ToMatUnsafe(this NDArray nDArray) =>
new Mat(nDArray.shape[0], nDArray.shape[1], nDArray.GetMatType(), new IntPtr(nDArray.Unsafe.Address));
public static MatType GetMatType(this NDArray nDArray)
{
int channels = nDArray.ndim == 3 ? nDArray.shape[2] : 1;
return nDArray.typecode switch
{
NPTypeCode.Int32 => channels == 1 ? MatType.CV_32SC1 :
channels == 2 ? MatType.CV_32SC2 :
throw new ArgumentException($"nDArray data type = {nDArray.typecode} & channels = {channels} is not supported"),
NPTypeCode.Float => channels == 1 ? MatType.CV_32FC1 :
throw new ArgumentException($"nDArray data type = {nDArray.typecode} & channels = {channels} is not supported"),
NPTypeCode.Double => channels == 1 ? MatType.CV_64FC1 :
throw new ArgumentException($"nDArray data type = {nDArray.typecode} & channels = {channels} is not supported"),
NPTypeCode.Byte => channels == 1 ? MatType.CV_8UC1 :
channels == 3 ? MatType.CV_8UC3 :
channels == 4 ? MatType.CV_8UC4 :
throw new ArgumentException($"nDArray data type = {nDArray.typecode} & channels = {channels} is not supported"),
_ => throw new ArgumentException($"nDArray data type = {nDArray.typecode} is not supported")
};
}You need to consider the number of channels to new Shape
There is a channels parameter in those functions above right ? Also I don't think you need to check if the channels parameter is equal to 1 because I don't think it will change anything ? I might be wrong tho
And I found another way to do it without touching the 'unsafe' code
In my opinion, the unsafe code is better because the unsafe code will create the NDArray wrap around the data of the input Mat with no memory copy (Marshal.Copy) involved (better performance). Of course, if performance is not a priority in your project the non-unsafe code will work too
There is a channels parameter in those functions above right ? Also I don't think you need to check if the channels parameter is equal to 1 because I don't think it will change anything ? I might be wrong tho
Yes, you can try these code.
var mat = new Mat(100, 100, MatType.CV_8UC1, Scalar.Black);
var array = mat.ToNDArray();
Console.WriteLine(array.ToString());
Console.WriteLine(array.ndim);If you don't consider the channels, It will create a wrong NDArray, and array.ndim will be equal to 3, not 2.
In my opinion, the unsafe code is better because the unsafe code will create the NDArray wrap around the data of the input Mat with no memory copy (Marshal.Copy) involved (better performance). Of course, if performance is not a priority in your project the non-unsafe code will work too
Although Marshal.Copy presents high performance, using pointer is definitely better.
That was a mistake sorry
If you don't consider the channels, It will create a wrong NDArray, and array.ndim will be equal to 3, not 2.
Can you try the code below ? I can't run these since I do not have my environment set up for SharpCV right now
var mat = new Mat(100, 100, MatType.CV_8UC1, Scalar.Black);
var array = mat.ToNDArray();
array = np.squeeze(array);
array = cv2.resize(array, (100, 100));
Console.WriteLine(array.ToString());
Console.WriteLine(array.ndim);