local variable 'data' referenced before assignment

[dm-druid]

Hello,
When I run '$python data_utils.py', it gives the error message like that:

File "corpus.py", line 63, in get_all_corpus
corpus.extend(data)
UnboundLocalError: local variable 'data' referenced before assignment

My python version is 2.7.10.
I don't know where the problem is.

[DevinZ1993]

Probably because _parse_corpus encounters some exception. Could you surround corpus.py line 61 with another try-catch and print out whatever exception that you catch?

[dm-druid]

I add the codes as you recommend: def get_all_corpus(): corpus = [] for raw in _corpus_list: json_file = os.path.join(data_dir, raw.replace('all', 'json').replace('txt', 'json')) try: with codecs.open(json_file, 'r', 'utf-8') as fin: data = json.load(fin) except IOError: try: data = _parse_corpus(os.path.join(raw_dir, raw), json_file) except Exception as e: print e finally: try: corpus.extend(data) except Exception as e: print e return corpus And the exceptions are shown as follow: $ python data_utils.py Building prefix dict from the default dictionary ... Loading model from cache /var/folders/64/1y8_58hs1nxf2hdlf96jtw900000gn/T/jieba.cache Loading model cost 0.469 seconds. Prefix dict has been built succesfully. Generating the vocabulary ... Parsing raw/qts_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qss_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qsc_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qtais_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/yuan.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/ming.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qing.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment The vocabulary has been built. Parsing raw/qts_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qss_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qsc_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qtais_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/yuan.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/ming.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qing.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qts_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qss_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qsc_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qtais_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/yuan.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/ming.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qing.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Start TextRank over the selected quatrains ... Parsing raw/qts_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qss_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qsc_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qtais_tab.txt ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/yuan.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/ming.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment Parsing raw/qing.all ... unichr() arg not in range(0x10000) (narrow Python build) local variable 'data' referenced before assignment [TextRank] Weighted graph has been built. [TextRank] Start iteration 0 ... Done (0/0) TextRank is done. Generating training data ... Training data is generated. Size of the training data: 0 Size of the keyword training data: 0

…

---- Is there something missing? I just clone the repo and run this command `python data_utils.py`. 2017-12-13 7:18 GMT+08:00 Nan Zuo <notifications@github.com>:

Probably because _parse_corpus encounters some exception. Could you surround corpus.py line 61 with another try-catch and print out whatever exception that you catch? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APFDeRBByk1VVtExfuwn-PRjjv0mR9Pfks5s_wm-gaJpZM4Q_JOm> .

[DevinZ1993]

That's quite similar to the previous issue here: unichr not supported by narrow python build. Hope this can help.

[dm-druid]

It works. So many thanks.