dmlc/MXNet.cpp

can't use dmlc::Stream::Create

Closed this issue · 5 comments

I'm trying use dmlc::Stream::Create to save the ndarray,but VS reports unresolved external symbol.

Will this function be added to MXNet.cpp

hjk41 commented

Nope. We will expose the DataIterator interface, which is already in c_api.h, instead of exposing the Stream interface directly. If you really need to use Stream directly, you better include the corresponding cpp file directly.

@hjk41 I'm little confuse about this. How do I use DataIterator interface to save ndarray,was it already in this C++ wrapper.

@zhubuntu
To save and load a NDArray, try

  /*!
  * \brief save a list of NDArrays to binary file.
  * \param file_name name of the binary file.
  * \param array_list a list of NDArrays.
  */
  static void Save(const std::string &file_name,
                   const std::vector<NDArray> &array_list);
  /*!
  * \brief Load list of NDArrays from binary file.
  * \param file_name name of the binary file.
  * \return a map from names to NDArrays.
  */
  static std::vector<NDArray> LoadToList(const std::string &file_name);

@zhangchen-qinyinghua Thanks for your help ,but i think the definition of static void Save(const std::string &file_name, const std::vector<NDArray> &array_list); is wrong.

void NDArray::Save(const std::string &file_name,
                   const std::vector<NDArray> &array_list) {
  std::vector<NDArrayHandle> args;
  std::vector<const char *> keys;
  CHECK_EQ(MXNDArraySave(file_name.c_str(), args.size(), args.data(), nullptr),
           0);
}

it didn't give the args value. And I change it to this

void NDArray::Save(const std::string &file_name,
                   const std::vector<NDArray> &array_list) {
  std::vector<NDArrayHandle> args;
  std::vector<const char *> keys;
  for (const auto &t : array_list) {
      args.push_back(t.GetHandle());
  }
  CHECK_EQ(MXNDArraySave(file_name.c_str(), args.size(), args.data(), nullptr),
           0);
}

My computer always report error when making PR,so maybe you can do this.

Thanks for your work.
It's fixed.