Prediction uses select column from the ground truth

Hi Wonseok:
thanks for your pretty work here and the issue is as following:
https://github.com/naver/sqlova/blob/master/train.py#L406
g_wvi_corenlp = get_g_wvi_corenlp(t)
this is using the "conds" in dev and if one want to predict from a plan sentence of english, he would not find the way to get something like "{"conds":[[0,0,"1998"]], "sel":1, "agg":0}".

the following url tells a similar problem on SQLnet
xiaojunxu/SQLNet#12

have i miss something or do you have a similar snippet of codes.

thank a lot

Hi @farsmile

To predict SQL from the plane text, you need to tokenize questions first by using StandforeCoreNLP, for example, by using following function.

sqlova/annotate_ws.py

Lines 18 to 34 in b7ce9ad

    
           def annotate(sentence, lower=True): 
        
               global client 
        
               if client is None: 
        
                   client = CoreNLPClient(default_annotators='ssplit,tokenize'.split(',')) 
        
               words, gloss, after = [], [], [] 
        
               for s in client.annotate(sentence): 
        
                   for t in s: 
        
                       words.append(t.word) 
        
                       gloss.append(t.originalText) 
        
                       after.append(t.after) 
        
               if lower: 
        
                   words = [w.lower() for w in words] 
        
               return { 
        
                   'gloss': gloss, 
        
                   'words': words, 
        
                   'after': after, 
        
                   }

After that, to generate SQL, the ground truth is not required. It is used in test function only to calculate loss and accuracies of dev set or test set. See below.

sqlova/train.py

Lines 432 to 441 in b7ce9ad

    
           s_sc, s_sa, s_wn, s_wc, s_wo, s_wv = model(wemb_n, l_n, wemb_h, l_hpu, l_hs) 
        
           # get loss & step 
        
           loss = Loss_sw_se(s_sc, s_sa, s_wn, s_wc, s_wo, s_wv, g_sc, g_sa, g_wn, g_wc, g_wo, g_wvi) 
        
           # prediction 
        
           pr_sc, pr_sa, pr_wn, pr_wc, pr_wo, pr_wvi = pred_sw_se(s_sc, s_sa, s_wn, s_wc, s_wo, s_wv, ) 
        
           pr_wv_str, pr_wv_str_wp = convert_pr_wvi_to_string(pr_wvi, nlu_t, nlu_tt, tt_to_t_idx, nlu) 
        
           # g_sql_i = generate_sql_i(g_sc, g_sa, g_wn, g_wc, g_wo, g_wv_str, nlu) 
        
           pr_sql_i = generate_sql_i(pr_sc, pr_sa, pr_wn, pr_wc, pr_wo, pr_wv_str, nlu)

Thanks.

Wonseok

Hi Wonseok:
it works again.
so much thanks!

您好Wonseok：
感谢您在这里的漂亮工作，问题如下：
https：//github.com/naver/sqlova/blob/master/train.py#L406
g_wvi_corenlp = get_g_wvi_corenlp(t)
这是在开发中使用“conds”，如果想要的话从英语的计划句中预测，他找不到像“{”conds“：[[0,0，”1998“]]，”sel“：1，”agg“：0}这样的方法。。

以下url告诉SQLnet xiaojunxu / SQLNet＃12上的类似问题
have i miss something or do you have a similar snippet of codes.
非常感谢

Hi Wonseok:
thanks for your pretty work here and the issue is as following:
https://github.com/naver/sqlova/blob/master/train.py#L406
g_wvi_corenlp = get_g_wvi_corenlp(t)
this is using the "conds" in dev and if one want to predict from a plan sentence of english, he would not find the way to get something like "{"conds":[[0,0,"1998"]], "sel":1, "agg":0}".

the following url tells a similar problem on SQLnet
xiaojunxu/SQLNet#12
have i miss something or do you have a similar snippet of codes.
thank a lot

same question, How did you change it?

	def annotate(sentence, lower=True):
	global client
	if client is None:
	client = CoreNLPClient(default_annotators='ssplit,tokenize'.split(','))
	words, gloss, after = [], [], []
	for s in client.annotate(sentence):
	for t in s:
	words.append(t.word)
	gloss.append(t.originalText)
	after.append(t.after)
	if lower:
	words = [w.lower() for w in words]
	return {
	'gloss': gloss,
	'words': words,
	'after': after,
	}

	s_sc, s_sa, s_wn, s_wc, s_wo, s_wv = model(wemb_n, l_n, wemb_h, l_hpu, l_hs)

	# get loss & step
	loss = Loss_sw_se(s_sc, s_sa, s_wn, s_wc, s_wo, s_wv, g_sc, g_sa, g_wn, g_wc, g_wo, g_wvi)

	# prediction
	pr_sc, pr_sa, pr_wn, pr_wc, pr_wo, pr_wvi = pred_sw_se(s_sc, s_sa, s_wn, s_wc, s_wo, s_wv, )
	pr_wv_str, pr_wv_str_wp = convert_pr_wvi_to_string(pr_wvi, nlu_t, nlu_tt, tt_to_t_idx, nlu)
	# g_sql_i = generate_sql_i(g_sc, g_sa, g_wn, g_wc, g_wo, g_wv_str, nlu)
	pr_sql_i = generate_sql_i(pr_sc, pr_sa, pr_wn, pr_wc, pr_wo, pr_wv_str, nlu)