Neargye/hello_tf_c_api

How to set session option?

Closed this issue · 8 comments

pango99 commented

Hi,how to set the session options using tensorflow c api? for example, how to set the gpu_memory_fraction? how to set the gpu number my program use?

Neargye commented

For RF 1.14.0 use TF_CreateConfig from c_api_experimental.h

// Create a serialized tensorflow.ConfigProto proto, where:
//
// a) ConfigProto.optimizer_options.global_jit_level is set to to ON_1 if
// `enable_xla_compilation` is non-zero, and OFF otherwise.
// b) ConfigProto.gpu_options.allow_growth is set to `gpu_memory_allow_growth`.
// c) ConfigProto.device_count is set to `num_cpu_devices`.
TF_CAPI_EXPORT extern TF_Buffer* TF_CreateConfig(
    unsigned char enable_xla_compilation, unsigned char gpu_memory_allow_growth,
    unsigned int num_cpu_devices);

and set this config use TF_SetConfig from c_api.h

// Set the config in TF_SessionOptions.options.
// config should be a serialized tensorflow.ConfigProto proto.
// If config was not parsed successfully as a ConfigProto, record the
// error information in *status.
TF_CAPI_EXPORT extern void TF_SetConfig(TF_SessionOptions* options,
                                        const void* proto, size_t proto_len,
                                        TF_Status* status);
pango99 commented

thanks your reply, but I still dont know how to do; I use tf-c-api only for inference,if computer has 2 GPUs(GPU0/GPU1), so how to config to force my program to run on GPU1 only?

other problem, from TF_CreateConfig(),how to set the "gpuMemFraction" param ? what's meaning of "enable_xla_compilation" param? I use tf-c-api only for inference,so can I set it to zero?

Neargye commented

Maybe @Xonxt could help.

Xonxt commented

@pango99
hi. For choosing a specific GPU in Tensorflow there exists a global solution: you have to create an environment variable CUDA_VISIBLE_DEVICES and list the numbers (id's) of the GPUs you want your Tensorflow to use. For example, if you have three GPUs, but you only want to use the first one and the third one, you set: CUDA_VISIBLE_DEVICES=0,2. In your case, if you have two, but you want to use GPU1, you set CUDA_VISIBLE_DEVICES=1.

Unfortunately, I don't know if this works with tf-c API. Try it out for yourself.

As for the memory fraction, I'd already created a function for it and Neargye has added it to his code as CreateSessionOptions().

  • 👍4
pango99 commented

@Xonxt
OK,I'll try it few days later,thank you!

msnh2012 commented

How to set "
sess = tf.Session( config= tf.ConfigProto(inter_op_parallelism_threads=1, intra_op_parallelism_threads=1))"
Thx a lot!
@Neargye
@Xonxt

Neargye commented

@msnh2012

uint8_t intra_op_parallelism_threads = 1;
uint8_t inter_op_parallelism_threads = 1;
uint8_t buf[] = {0x10,i ntra_op_parallelism_threads, 0x28, inter_op_parallelism_threads};
TF_SetConfig(sess_opts, buf, sizeof(buf), status);

See tensorflow/tensorflow#13853

apivovarov commented

@msnh2012 @Neargye to get proto parameter for TF_SetConfig we can use tensorflow/core/protobuf/config.pb.h and tensorflow::ConfigProto class
The implementation is included to libtensorflow.so

Code example https://github.com/apivovarov/TF_C_API/blob/master/config.cc

build command:

g++ -std=c++11 -o config.o -c -Itensorflow/bazel-tensorflow/external/com_google_protobuf/src -Itensorflow/bazel-bin -Ilibtensorflow/include config.cc
g++ -std=c++11 -o config config.o -Llibtensorflow/lib -ltensorflow -ltensorflow_framework 
LD_LIBRARY_PATH=libtensorflow/lib: ./config
0x10,0x2,0x28,0x3,0x32,0xb,0x9,0x9a,0x99,0x99,0x99,0x99,0x99,0xb9,0x3f,0x20,0x1

TF_SetConfig API:
https://github.com/tensorflow/tensorflow/blob/r1.15/tensorflow/c/c_api.h#L147

  • 👍2