Thursday, May 11, 2017

Tensorflow custom operation - problem with word2vec operation

This is just a quick note for myself. One beautiful afternoon I got a below error:

tensorflow.python.framework.errors_impl.NotFoundError: 
models/tutorials/embedding/word2vec_ops.so: 
undefined symbol: _ZN10tensorflow16ReadFileToStringEPNS_3EnvERKSsPSs
The distributed package of Tensorflow uses gcc 4.
If you compile from sources on more or less up to date Ubuntu or Debian, you probably will have gcc 5 or newer installed.
gcc 5 and gcc 4 ABI are not compatible, thus your Tensorflow and operations MUST be compiled with same ABI.
You have two choices either compile the Tensorflow with old ABI support by providing
-cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"
flag to bazel build or ensure your operations are compiled with new ABI.
You can also force gcc5 to use old ABI (source):

TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
g++ -std=c++11 -shared word2vec_ops.cc word2vec_kernels.cc \
  -o word2vec_ops.so -fPIC -I $TF_INC -O2 -D_GLIBCXX_USE_CXX11_ABI=0

More could be found in Tensorflow doc: https://www.tensorflow.org/install/install_sources#build_the_pip_package

References: