How to save a tensor to a image

[ypflll]

I am trying to convert my code from python to c++ and your repo helps me a lot.

However, I am still confused with the data type of tf tensor, eigen tensor and cv mat.
My model inputs a image with shape [1,256,256,3] and get a output mask with shape [1,256,256,1]. When I get the output tf tensor, how to save it?
I tried to concert it to a cv mat and using cv::imwrite() but failed. Any suggestion?

[PatWie]

Tensors are written in row-major as a continous array. You can always access them by T(i1, i2).

[PatWie]

See new example.

[ypflll]

Hi, thanks a lot for your code.
However, it doesn't work for me.
I get a tensor with size [1,224,224,1], type is float32.
Follow you rcode, I use this:

TF_CHECK_OK(sess_seg->Run(feed_dict, {"segmentation_result"}, {}, &seg_result));

  tensorflow::Tensor output = seg_result[0];
  //std::cout << "output " << output << std::endl;
  float *data_ptr = output.flat<float>().data();

  std::cout << "Output TensorShape ["
            << output.shape().dim_size(0) << ", "
            << output.shape().dim_size(1) << ", "
            << output.shape().dim_size(2) << ", "
            << output.shape().dim_size(3) << "]" << std::endl;

  //Mat result_seg(224, 224, CV_8U);
  Mat result_seg;
  result_seg.create(output.dim_size(1), output.dim_size(2), CV_32FC3);
  copy_n((char*) data_ptr , output.shape().num_elements() * sizeof(float), (char*) result_seg.data);
  cout << output.shape().num_elements() * sizeof(float) << endl;

It gets a result like this:

I use this code to get the right image:

TF_CHECK_OK(sess_seg->Run(feed_dict, {"segmentation_result"}, {}, &seg_result));

  tensorflow::TTypes<float>::Flat output = seg_result[0].flat<float>();
  //std::cout << "output " << output << std::endl;
  float *data_ptr = output.data();

  //fill data to a cv rotMatrix 
  cv::Mat_<float> result_seg = cv::Mat_<float>::ones(224,224);
  for (int x = 0; x < 224; x++) {
    for (int y = 0; y <224; y++) {
      result_seg.at<float>(x,y) = round(*(data_ptr+224*x+y));
    }
  }
  cv::imwrite("/xxx/seg_result.jpg", result_seg*255);
  Mat result_seg1;
  result_seg1 = imread("/xxx/seg_result.jpg");

I think it's a problem of datatype, cause my tensor gives out float32, but the image need uin8. But I try to set the datatype all to uint8 or float32, still not work.

[ypflll]

This also works for me:
cv::Mat rotMatrix(seg_result[0].dim_size(1), seg_result[0].dim_size(2), CV_32FC1, seg_result[0].flat<float>().data());

[PatWie]

This is not an issue with the implementation and depends on your actual image channels (OpenCV).

[ypflll]

Yeah, I falsely read 3 channels when the output tensor has 1 channel.

BTW, why do you convert float to uchar in copy_n (seems that copy type float also works):
std::copy_n((char*) image_float_data, image_shape.num_elements() * sizeof(float), const_cast<char*>(image_tensor.tensor_data().data()));
Something about the big-endian and little endian?

[deeptf]

This also works for me:
cv::Mat rotMatrix(seg_result[0].dim_size(1), seg_result[0].dim_size(2), CV_32FC1, seg_result[0].flat<float>().data());

@PatWie : Which namespace does rotMatrix belongs to? I do not see such a function in OpenCV.

[PatWie]

rotMatrix is the name of the new matrix (the ctor is called here).

[deeptf]

Thanks for the clarification. Would this still hold for RGB Image? I see a weird image. Although I have changed CV_32FC1 to CV_32FC3

[PatWie]

TensorFlow is likely to store the image as rrrrr......rrrrgggggg......ggggbbbbb.....bbbbb while OpenCV stores the values as bgrbgrbgrbgr....

[deeptf]

Do you have any suggestion on how I can achieve this?

[PatWie]

Write a loop and copy the values.

[Aeroxander]

I wouldn't write a loop and copy the values, I converted an ARGB to BGR with switching the channels through OpenCV, no looping necessary (for PyTorch at least).

[PatWie]

Then OpenCV does the loop for you ;-)

[duocang]

Remind:
I have tried with different color type, only type 4 works for me.

session->Run({ {input_layers, input_tensor} }, { output_layers[0], output_layers[1] }, {}, &outputs);
pred_labels = outputs[1];

int* p = pred_labels.flat<int>().data();
cv::Mat rotMatrix(pred_labels.dim_size(0), pred_labels.dim_size(1), 4, p);

imwrite(song, rotMatrix * 255);

[ucesfpa]

My solution with Tf 2.4
cv2.imwrite('img0.png',tf.image.convert_image_dtype(tensor_image, tf.uint8, saturate=True).numpy())