go-ego/gse

gse dict lose efficacy when set an empty string

CharlieWangKD opened this issue · 0 comments

  • Gse version: v0.80.2
  • Go version: 1.19
func main() {
	seg := new(gse.Segmenter)
	seg.Dict = gse.NewDict()
	seg.Init()
	seg.AddToken("bj", 100, "n")
	fmt.Println(seg.Dictionary())
	fmt.Println(seg.Find("bj"))
	seg.AddToken("", 100, "n")
	fmt.Println(seg.Dictionary())
	fmt.Println(seg.Find("bj"))
}
// output:
// &{0xc000140000 1 [{[[98 106]] 100 n 0 []}] 100}
// 100 n true
// &{0xc000140000 1 [{[[98 106]] 100 n 0 []} {[] 100 n 0 []}] 200}
// 0  false

Description

The Find func of dict will lose efficacy when I set an empty string, the instance will be unable to cutting string.