absl flags/tf.app.flags
AndrewLischishin opened this issue · 5 comments
AndrewLischishin commented
Hi there! I've tried for a while to run your code, but still have some flag problems, so here what I get:
this is what I get if I just 'make' in nips2016:
jupyter nbconvert --inplace --execute --ExecutePreprocessor.timeout=-1 20news.ipynb
[NbConvertApp] Converting notebook 20news.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Error while converting '20news.ipynb'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/nbconvert/nbconvertapp.py", line 393, in export_single_notebook
output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
File "/usr/local/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 174, in from_filename
return self.from_file(f, resources=resources, **kw)
File "/usr/local/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 192, in from_file
return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
File "/usr/local/lib/python3.6/site-packages/nbconvert/exporters/notebook.py", line 31, in from_notebook_node
nb_copy, resources = super(NotebookExporter, self).from_notebook_node(nb, resources, **kw)
File "/usr/local/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 134, in from_notebook_node
nb_copy, resources = self._preprocess(nb_copy, resources)
File "/usr/local/lib/python3.6/site-packages/nbconvert/exporters/exporter.py", line 311, in _preprocess
nbc, resc = preprocessor(nbc, resc)
File "/usr/local/lib/python3.6/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
return self.preprocess(nb, resources)
File "/usr/local/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 262, in preprocess
nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
File "/usr/local/lib/python3.6/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
File "/usr/local/lib/python3.6/site-packages/nbconvert/preprocessors/execute.py", line 286, in preprocess_cell
raise CellExecutionError.from_cell_and_msg(cell, out)
nbconvert.preprocessors.execute.CellExecutionError: An error occurred while executing the following cell:
------------------
# Fetch dataset. Scikit-learn already performs some cleaning.
remove = ('headers','footers','quotes') # (), ('headers') or ('headers','footers','quotes')
train = utils.Text20News(data_home=FLAGS.dir_data, subset='train', remove=remove)
#Pre-processing: transform everything to a-z and whitespace.
print(train.show_document(1)[:400])
train.clean_text(num='substitute')
# Analyzing / tokenizing: transform documents to bags-of-words.
#stop_words = set(sklearn.feature_extraction.text.ENGLISH_STOP_WORDS)
# Or stop words from NLTK.
# Add e.g. don, ve.
train.vectorize(stop_words='english')
print(train.show_document(1)[:400])
------------------
------------------------------------------------------------------------
UnrecognizedFlagError Traceback (most recent call last)
<ipython-input-3-0b079301697c> in <module>()
1 # Fetch dataset. Scikit-learn already performs some cleaning.
2 remove = ('headers','footers','quotes') # (), ('headers') or ('headers','footers','quotes')
----> 3 train = utils.Text20News(data_home=FLAGS.dir_data, subset='train', remove=remove)
4
5 # Pre-processing: transform everything to a-z and whitespace.
/usr/local/lib/python3.6/site-packages/tensorflow/python/platform/flags.py in __getattr__(self, name)
82 # a flag.
83 if not wrapped.is_parsed():
---> 84 wrapped(_sys.argv)
85 return wrapped.__getattr__(name)
86
/usr/local/lib/python3.6/site-packages/absl/flags/_flagvalues.py in __call__(self, argv, known_only)
628 suggestions = _helpers.get_flag_suggestions(name, list(self))
629 raise _exceptions.UnrecognizedFlagError(
--> 630 name, value, suggestions=suggestions)
631
632 self.mark_as_parsed()
UnrecognizedFlagError: Unknown command line flag 'f'
UnrecognizedFlagError: Unknown command line flag 'f'
I tried to solve it myself and what I found was, that tensorflow uses now absl for flags, so I tried to do this:
from absl import flags
from absl import logging
from absl import app
FLAGS = flags.FLAGS
instead of
#flags = tf.app.flags
#FLAGS = flags.FLAGS
but still get another error
# Graphs.
flags.DEFINE_integer('number_edges', 8, 'Graph: minimum number of edges per vertex.')
flags.DEFINE_string('metric', 'euclidean', 'Graph: similarity measure (between features).')
# TODO: change cgcnn for combinatorial Laplacians.
flags.DEFINE_bool('normalized_laplacian', True, 'Graph Laplacian: normalized.')
flags.DEFINE_integer('coarsening_levels', 4, 'Number of coarsened graphs.')
#Directories.
flags.DEFINE_string('dir_data', os.path.join('..', 'data', 'mnist'), 'Directory to store data.')`
def grid_graph(m, corners=False):
z = graph.grid(m)
dist, idx = graph.distance_sklearn_metrics(z, k=FLAGS.number_edges, metric=FLAGS.metric)
A = graph.adjacency(dist, idx)
# Connections are only vertical or horizontal on the grid.
# Corner vertices are connected to 2 neightbors only.
if corners:
import scipy.sparse
A = A.toarray()
A[A < A.max()/1.5] = 0
A = scipy.sparse.csr_matrix(A)
print('{} edges'.format(A.nnz))
print("{} > {} edges".format(A.nnz//2, FLAGS.number_edges*m**2//2))
return A
t_start = time.process_time()
A = grid_graph(28, corners=False)
A = graph.replace_random_edges(A, 0)
graphs, perm = coarsening.coarsen(A, levels=FLAGS.coarsening_levels, self_connections=False)
L = [graph.laplacian(A, normalized=True) for A in graphs]
print('Execution time: {:.2f}s'.format(time.process_time() - t_start))
graph.plot_spectrum(L)
del A
---------------------------------------------------------------------------
UnparsedFlagAccessError Traceback (most recent call last)
<ipython-input-3-b674f783dcfd> in <module>()
17
18 t_start = time.process_time()
---> 19 A = grid_graph(28, corners=False)
20 A = graph.replace_random_edges(A, 0)
21 graphs, perm = coarsening.coarsen(A, levels=FLAGS.coarsening_levels, self_connections=False)
<ipython-input-3-b674f783dcfd> in grid_graph(m, corners)
1 def grid_graph(m, corners=False):
2 z = graph.grid(m)
----> 3 dist, idx = graph.distance_sklearn_metrics(z, k=FLAGS.number_edges, metric=FLAGS.metric)
4 A = graph.adjacency(dist, idx)
5
/usr/local/lib/python3.6/site-packages/absl/flags/_flagvalues.py in __getattr__(self, name)
486 # get too much noise.
487 logging.error(error_message)
--> 488 raise _exceptions.UnparsedFlagAccessError(error_message)
489
490 def __setattr__(self, name, value):
UnparsedFlagAccessError: Trying to access flag --number_edges before flags were parsed.
So, maybe I just overlook something, but still stuck in it.
hurun commented
i also stuck this problem,and not solve it still now.
AndrewLischishin commented
So, this problem is also discussed here, and it pops out because of using jupyter notebook.
In order to be able to run mnist.py(or any other) just convert the .ipynb file into .py, for example with this:
$ jupyter nbconvert --to script [YOUR_NOTEBOOK].ipynb
then you can just normally execute with:
python<version> file.py
if some matplotlib problems pop out check this, maybe it will answer your questions, or this.
I am using MacBook Pro 2014mid, macOS High Sierra, Version 10.13.3, python 3.6.4, tensorflow 1.6.0. and I didn't instal tensorflow-gpu from requirements.txt(brought some errors in).
The first solution worked for me, when the second one just brought some blocking while executing.
If someone found other, more simple, solution or maybe it is just correct as it is, please comment.
prateeky2806 commented
I am facing the same issue, after converting to .py file there are lots of errors to resolve. Isn't there a way to fix this in jupyter notebook only?
TMorville commented
To anyone having this issue in the notebook, add tf.app.flags.DEFINE_string('f', '', 'kernel') before defining your flags.
E.g. for rcv1
flags = tf.app.flags
FLAGS = flags.FLAGS
#
tf.app.flags.DEFINE_string('f', '', 'kernel')
# Graphs.
flags.DEFINE_integer('number_edges', 16, 'Graph: minimum number of edges per vertex.')
flags.DEFINE_string('metric', 'cosine', 'Graph: similarity measure (between features).')
# TODO: change cgcnn for combinatorial Laplacians.
flags.DEFINE_bool('normalized_laplacian', True, 'Graph Laplacian: normalized.')
flags.DEFINE_integer('coarsening_levels', 0, 'Number of coarsened graphs.')
flags.DEFINE_string('dir_data', os.path.join('data', 'rcv1'), 'Directory to store data.')
flags.DEFINE_integer('val_size', 400, 'Size of the validation set.')
mdeff commented
That's probably due to a TensorFlow update. The original code should work with tensorflow-gpu==1.1.0 (the version I used to develop the code).
If the fix (adding tf.app.flags.DEFINE_string('f', '', 'kernel')) also works with tensorflow-gpu==1.1.0 I'd accept a PR.