After infra reorg this spring, editdistance info comes up as 0
Trondtr opened this issue · 2 comments
Trondtr commented
After make clean in lang-fkv/tools/spellcheckers/, I get the following for make V=1
uit-mac-443 lang-fkv (main)$ make V=1
*** Compiling fkv - Kven Finnish. ***
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh '/Users/ttr000/git/giellalt/lang-fkv/build-aux/missing' autoconf
Making all in .
*** Compiling fkv - Kven Finnish. ***
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh '/Users/ttr000/git/giellalt/lang-fkv/build-aux/missing' autoconf
make[1]: Nothing to be done for `all-am'.
Making all in src
Making all in fst
Making all in morphology
make[3]: Nothing to be done for `all'.
Making all in filters
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt area-tags.txt.tmp Area/
if test -f area-tags.txt ; then \
if ! diff -q area-tags.txt area-tags.txt.tmp ; then \
mv -f area-tags.txt.tmp area-tags.txt ; \
else \
rm -f area-tags.txt.tmp ; \
fi ;\
else \
mv -f area-tags.txt.tmp area-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt derivation-tags.txt.tmp Der/
if test -f derivation-tags.txt ; then \
if ! diff -q derivation-tags.txt derivation-tags.txt.tmp ; then \
mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
else \
rm -f derivation-tags.txt.tmp ; \
fi ;\
else \
mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt dialect-tags.txt.tmp Dial/
if test -f dialect-tags.txt ; then \
if ! diff -q dialect-tags.txt dialect-tags.txt.tmp ; then \
mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
else \
rm -f dialect-tags.txt.tmp ; \
fi ;\
else \
mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt error-tags.txt.tmp Err/
if test -f error-tags.txt ; then \
if ! diff -q error-tags.txt error-tags.txt.tmp ; then \
mv -f error-tags.txt.tmp error-tags.txt ; \
else \
rm -f error-tags.txt.tmp ; \
fi ;\
else \
mv -f error-tags.txt.tmp error-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt grammatical-tags.txt.tmp Gram/
if test -f grammatical-tags.txt ; then \
if ! diff -q grammatical-tags.txt grammatical-tags.txt.tmp ; then \
mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
else \
rm -f grammatical-tags.txt.tmp ; \
fi ;\
else \
mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt orig_lang-tags.txt.tmp OLang/
if test -f orig_lang-tags.txt ; then \
if ! diff -q orig_lang-tags.txt orig_lang-tags.txt.tmp ; then \
mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
else \
rm -f orig_lang-tags.txt.tmp ; \
fi ;\
else \
mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt semantic-tags.txt.tmp Sem/
if test -f semantic-tags.txt ; then \
if ! diff -q semantic-tags.txt semantic-tags.txt.tmp ; then \
mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
else \
rm -f semantic-tags.txt.tmp ; \
fi ;\
else \
mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt usage-tags.txt.tmp Use/
if test -f usage-tags.txt ; then \
if ! diff -q usage-tags.txt usage-tags.txt.tmp ; then \
mv -f usage-tags.txt.tmp usage-tags.txt ; \
else \
rm -f usage-tags.txt.tmp ; \
fi ;\
else \
mv -f usage-tags.txt.tmp usage-tags.txt ; \
fi
/Library/Developer/CommandLineTools/usr/bin/make all-am
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt orig_lang-tags.txt.tmp OLang/
if test -f orig_lang-tags.txt ; then \
if ! diff -q orig_lang-tags.txt orig_lang-tags.txt.tmp ; then \
mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
else \
rm -f orig_lang-tags.txt.tmp ; \
fi ;\
else \
mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt area-tags.txt.tmp Area/
if test -f area-tags.txt ; then \
if ! diff -q area-tags.txt area-tags.txt.tmp ; then \
mv -f area-tags.txt.tmp area-tags.txt ; \
else \
rm -f area-tags.txt.tmp ; \
fi ;\
else \
mv -f area-tags.txt.tmp area-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt derivation-tags.txt.tmp Der/
if test -f derivation-tags.txt ; then \
if ! diff -q derivation-tags.txt derivation-tags.txt.tmp ; then \
mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
else \
rm -f derivation-tags.txt.tmp ; \
fi ;\
else \
mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt dialect-tags.txt.tmp Dial/
if test -f dialect-tags.txt ; then \
if ! diff -q dialect-tags.txt dialect-tags.txt.tmp ; then \
mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
else \
rm -f dialect-tags.txt.tmp ; \
fi ;\
else \
mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt error-tags.txt.tmp Err/
if test -f error-tags.txt ; then \
if ! diff -q error-tags.txt error-tags.txt.tmp ; then \
mv -f error-tags.txt.tmp error-tags.txt ; \
else \
rm -f error-tags.txt.tmp ; \
fi ;\
else \
mv -f error-tags.txt.tmp error-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt grammatical-tags.txt.tmp Gram/
if test -f grammatical-tags.txt ; then \
if ! diff -q grammatical-tags.txt grammatical-tags.txt.tmp ; then \
mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
else \
rm -f grammatical-tags.txt.tmp ; \
fi ;\
else \
mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt semantic-tags.txt.tmp Sem/
if test -f semantic-tags.txt ; then \
if ! diff -q semantic-tags.txt semantic-tags.txt.tmp ; then \
mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
else \
rm -f semantic-tags.txt.tmp ; \
fi ;\
else \
mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt usage-tags.txt.tmp Use/
if test -f usage-tags.txt ; then \
if ! diff -q usage-tags.txt usage-tags.txt.tmp ; then \
mv -f usage-tags.txt.tmp usage-tags.txt ; \
else \
rm -f usage-tags.txt.tmp ; \
fi ;\
else \
mv -f usage-tags.txt.tmp usage-tags.txt ; \
fi
Making all in phonetics
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in tests
make[4]: Nothing to be done for `all'.
Making all in syllabification
make[3]: Nothing to be done for `all'.
Making all in orthography
make[3]: Nothing to be done for `all'.
Making all in transcriptions
make[3]: Nothing to be done for `all'.
Making all in tagsets
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in cg3
make[2]: Nothing to be done for `all'.
make[2]: Nothing to be done for `all-am'.
Making all in tools
Making all in tokenisers
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in tests
make[3]: Nothing to be done for `all'.
Making all in analysers
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in shellscripts
make[2]: Nothing to be done for `all'.
Making all in spellcheckers
Making all in filters
cp -f /Users/ttr000/git/giellalt/lang-fkv/./../giella-core/fst-filters/remove-word-boundary.regex remove-word-boundary.regex
"/usr/local/bin/hfst-regexp2fst" \
--format=openfst-tropical \
"--xerox-composition=ON" -v -S -i remove-word-boundary.regex -o remove-word-boundary.hfst
Reading from remove-word-boundary.regex, writing to remove-word-boundary.hfst
Compiling expression #1
Compiling expression #2
cp -f /Users/ttr000/git/giellalt/lang-fkv/src/fst/filters/remove-usage-tags.regex remove-usage-tags.regex
"/usr/local/bin/hfst-regexp2fst" \
--format=openfst-tropical \
"--xerox-composition=ON" -v -S -i remove-usage-tags.regex -o remove-usage-tags.hfst
Reading from remove-usage-tags.regex, writing to remove-usage-tags.hfst
Compiling expression #1
Compiling expression #2
cp -f /Users/ttr000/git/giellalt/lang-fkv/src/fst/filters/remove-error-strings.regex remove-error-strings.regex
"/usr/local/bin/hfst-regexp2fst" \
--format=openfst-tropical \
"--xerox-composition=ON" -v -S -i remove-error-strings.regex -o remove-error-strings.hfst
Reading from remove-error-strings.regex, writing to remove-error-strings.hfst
Compiling expression #1
Compiling expression #2
Making all in weights
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/clean_weight_corpus.sh spellercorpus.raw.txt spellercorpus.clean.txt
Making all in neural
make[3]: Nothing to be done for `all'.
Making all in .
/usr/local/bin/gmkdir -p `dirname .generated/.stamp`
touch .generated/.stamp
/usr/bin/printf "read regex \
@\"../../src/fst/filters/remove-usage_except_speller-tags.hfst\" \
.o. @\"../../src/fst/filters/remove-orig_lang-tags.hfst\" \
.o. @\"../../src/fst/filters/remove-CLB-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-error-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-MinusSpell-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-PUNCT-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-use_marg-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-Use_minus_PMatch-tags.hfst\" \
.o. @\"../../src/fst/filters/remove-Use_minusGC-tags.hfst\" \
.o. @\"../../src/fst/filters/remove-Use_GC-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-Use_PMatch-strings.hfst\" \
.o. @\"../../src/fst/filters/remove-mwe-tags.hfst\" \
.o. @\"../../src/fst/filters/remove-use_mt-strings.hfst\" \
.o. @\"../../src/fst/analyser-raw-gt-desc.hfst\" \
.o. @\"../../src/fst/orthography/downcase-derived_proper-strings.compose.hfst\" \
;\n\
save stack .generated/generator-speller-gt-norm.tmp.hfst\n\
quit\n" | /usr/local/bin/hfst-xfst -p -v --format=foma
Using foma as output handler
Reading from standard input...
warning: both composition arguments contain flag diacritics that are not harmonized
? bytes. 136976 states, 335416 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-speller-gt-norm.tmp.hfst .generated/generator-speller-gt-norm.hfst
cp .generated/generator-speller-gt-norm.hfst generator-speller-gt-norm.hfst
/usr/bin/printf "read regex \
\
@\"generator-speller-gt-norm.hfst\" \
.o. @\"../../src/fst/filters/remove-hyphenation-marks.hfst\" \
.o. @\"../../src/fst/filters/remove-infl_deriv-borders.hfst\" \
; \n\
twosided flag-diacritics\n\
save stack .generated/generator-fstspeller-gt-norm.tmp.hfst\n\
quit\n" | /usr/local/bin/hfst-xfst -p -v --format=foma
Using foma as output handler
Reading from standard input...
? bytes. 135461 states, 335095 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-fstspeller-gt-norm.tmp.hfst .generated/generator-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-fst2fst -f openfst-tropical .generated/generator-fstspeller-gt-norm.hfst -o .generated/generator-mobilespeller-gt-norm-base.hfst
/usr/local/bin/hfst-reweight -v \
-T ./weights/tags.reweight --arcs-only -i .generated/generator-mobilespeller-gt-norm-base.hfst \
-o .generated/generator-mobilespeller-gt-norm-tag_weighted.hfst
Reading from .generated/generator-mobilespeller-gt-norm-base.hfst, writing to .generated/generator-mobilespeller-gt-norm-tag_weighted.hfst
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 0.000000
only on arc weights, no end states
Reweighting convert(UNNAMED)...
Reading reweights from ./weights/tags.reweight
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 2.000000 for symbol +Cmp
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 1.000000 for symbol +Der/minen
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 50000.000000 for symbol +Use/SpellNoSugg
/usr/bin/printf "\
set encode-weights ON \n\
read regex \
@\"filters/remove-usage-tags.hfst\" \
.o. @\".generated/generator-mobilespeller-gt-norm-tag_weighted.hfst\" \
.o. @\"filters/remove-word-boundary.hfst\" \
; \n\
save stack .generated/generator-mobilespeller-gt-norm.tmp.hfst\n\
quit\n" | /usr/local/bin/hfst-xfst -p -v
Using default output format OpenFst with tropical weight class
Using OpenFst's tropical weights as output
Reading from standard input...
variable encode-weights = ON
hfst[0]: ? bytes. 131662 states, 334472 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-mobilespeller-gt-norm.tmp.hfst .generated/generator-mobilespeller-gt-norm.hfst
cp .generated/generator-mobilespeller-gt-norm.hfst generator-mobilespeller-gt-norm.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/make-hfstspeller-version-easter-egg.sh \
fkv \
../.. \
0.1.1 \
"mobile" \
default \
> easteregg.default.mobile.txt
echo "?*" | /usr/local/bin/hfst-regexp2fst -o .generated/anystar.hfst
cp .generated/anystar.hfst anystar.hfst
sed -e 's/^/nuvviDspeller:/' < easteregg.default.mobile.txt \
| sed = \
| sed 'N;s/\n/ /' \
| perl -pe 's/(.)\t(.+)/\2\t\1/' \
> easteregg.default.mobile.suggtxt
grep -h -v '^#' words.default.txt easteregg.default.mobile.suggtxt | grep -v '^$' \
| /usr/local/bin/hfst-strings2fst -j \
-o words.default.mobile.hfst
grep -v '^#' strings.default.txt | grep -v '^$' | cut -f1-2 \
| /usr/local/bin/hfst-strings2fst -j \
| /usr/local/bin/hfst-concatenate anystar.hfst - \
| /usr/local/bin/hfst-concatenate - anystar.hfst \
-o strings.txt.default.mobile.hfst
cp -f strings.txt.default.mobile.hfst strings.all.default.mobile.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/editdist.py \
--verbose \
\
--epsilon='@0@' \
--default-weight=10 \
--regex \
--input=editdist.default.txt \
--output-file=editdist.default.mobile.regex \
/usr/local/bin/hfst-regexp2fst -S -i editdist.default.mobile.regex\
--format=openfst-tropical \
-o .generated/editdist.default.mobile.hfst
cp .generated/editdist.default.mobile.hfst editdist.default.mobile.hfst
/usr/local/bin/hfst-disjunct strings.all.default.mobile.hfst editdist.default.mobile.hfst \
| /usr/local/bin/hfst-repeat -f 1 -t 2 \
-o editdist.all.default.mobile.hfst
grep -v '^#' final_strings.default.txt | grep -v '^$' | cut -f1-2 \
| /usr/local/bin/hfst-strings2fst -j \
--format=openfst-tropical \
-o final_strings.txt.default.mobile.hfst
cp -f final_strings.txt.default.mobile.hfst final_strings.all.default.mobile.hfst
printf "\
[ @\"words.default.mobile.hfst\" \
| \
[ \
\
@\"editdist.all.default.mobile.hfst\" \
( @\"final_strings.all.default.mobile.hfst\" ) \
] \
];" \
| /usr/local/bin/hfst-regexp2fst -S -E \
| /usr/local/bin/hfst-push-weights --push=initial \
| /usr/local/bin/hfst-fst2fst -f olw \
-o errmodel.default.mobile.hfst
/usr/local/bin/hfst-strings2fst -j < easteregg.default.mobile.txt \
| /usr/local/bin/hfst-project --project=lower > easteregg.default.mobile.hfst
/usr/local/bin/hfst-project \
-v --project=lower < generator-mobilespeller-gt-norm.hfst \
| /usr/local/bin/hfst-remove-epsilons -v \
| /usr/local/bin/hfst-disjunct -v - easteregg.default.mobile.hfst \
| /usr/local/bin/hfst-push-weights -v --push=initial \
| /usr/local/bin/hfst-fst2fst -v -f olw \
-o acceptor.default.mobile.hfst
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from - and easteregg.default.mobile.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to acceptor.default.mobile.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Projecting second <stdin>...
Removing epsilons project-2nd(UNNAMED)...
Disjuncting remove-epsilons(project-2nd(UNNAMED)) and project-2nd(strings(?))...
Pushing towards start union(remove-epsilons(project-2nd(UNNAMED)), project-2nd(strings(?)))...
Converting push-weights-i(union(remove-epsilons(project-2nd(UNNAMED)), project-2nd(strings(?))))...
rm -f fkv-mobile.zhfst
/usr/local/bin/gmkdir -p build/fkv-mobile.zhfst
rm -f build/fkv-mobile.zhfst/*
cp index.mobile.xml build/fkv-mobile.zhfst/index.xml
cp acceptor.default.mobile.hfst build/fkv-mobile.zhfst/acceptor.default.hfst
cp errmodel.default.mobile.hfst build/fkv-mobile.zhfst/errmodel.default.hfst
cd build/fkv-mobile.zhfst/ && /usr/bin/zip -0 ../../fkv-mobile.zhfst *
adding: acceptor.default.hfst (stored 0%)
adding: errmodel.default.hfst (stored 0%)
adding: index.xml (stored 0%)
/usr/local/bin/gmkdir -p 3
cp -f fkv-mobile.zhfst 3/
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/predict.py \
--verbose \
--epsilon='@0@' \
--default-weight=10 \
--regex \
--input=editdist.default.txt \
--output=predict.default.regex
being verbose...
reading from editdist.default.txt...
writing to predict.default.regex...
"/usr/local/bin/hfst-regexp2fst" --format=foma \
-v -S predict.default.regex -o predict.default.hfst
Reading from predict.default.regex, writing to predict.default.hfst
Compiling expression #1
Compiling expression #2
hfst-fst2fst -f olw -v -i predict.default.hfst -o errmodel.default.predict.hfst
Reading from predict.default.hfst, writing to errmodel.default.predict.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Converting xre(?)...
/usr/local/bin/hfst-fst2fst -f olw -i acceptor.default.mobile.hfst -o acceptor.default.predict.hfst
rm -f fkv-x-predict-mobile.zhfst
/usr/local/bin/gmkdir -p build/fkv-x-predict-mobile.zhfst
rm -f build/fkv-x-predict-mobile.zhfst/*
cp index.predict.xml build/fkv-x-predict-mobile.zhfst/index.xml
cp acceptor.default.predict.hfst build/fkv-x-predict-mobile.zhfst/acceptor.default.hfst
cp errmodel.default.predict.hfst build/fkv-x-predict-mobile.zhfst/errmodel.default.hfst
cd build/fkv-x-predict-mobile.zhfst/ && /usr/bin/zip -0 ../../fkv-x-predict-mobile.zhfst *
adding: acceptor.default.hfst (stored 0%)
adding: errmodel.default.hfst (stored 0%)
adding: index.xml (stored 0%)
/usr/local/bin/gmkdir -p 3
cp -f fkv-x-predict-mobile.zhfst 3/
/usr/local/bin/hfst-invert -v -i .generated/generator-speller-gt-norm.hfst -o .generated/analyser-speller-gt-norm.hfst
Reading from .generated/generator-speller-gt-norm.hfst, writing to .generated/analyser-speller-gt-norm.hfst
Inverting .generated/generator-speller-gt-norm.hfst...
cp .generated/analyser-speller-gt-norm.hfst analyser-speller-gt-norm.hfst
cp .generated/generator-fstspeller-gt-norm.hfst generator-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-invert -v -i .generated/generator-fstspeller-gt-norm.hfst -o .generated/analyser-fstspeller-gt-norm.hfst
Reading from .generated/generator-fstspeller-gt-norm.hfst, writing to .generated/analyser-fstspeller-gt-norm.hfst
Inverting .generated/generator-fstspeller-gt-norm.hfst...
cp .generated/analyser-fstspeller-gt-norm.hfst analyser-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-fst2fst --format=openfst-tropical -i .generated/generator-fstspeller-gt-norm.hfst -o .generated/generator-desktopspeller-gt-norm-base.hfst
/usr/local/bin/hfst-reweight -v \
-T ./weights/tags.reweight --arcs-only -i .generated/generator-desktopspeller-gt-norm-base.hfst \
-o .generated/generator-desktopspeller-gt-norm-tag_weighted.hfst
Reading from .generated/generator-desktopspeller-gt-norm-base.hfst, writing to .generated/generator-desktopspeller-gt-norm-tag_weighted.hfst
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 0.000000
only on arc weights, no end states
Reweighting convert(UNNAMED)...
Reading reweights from ./weights/tags.reweight
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 2.000000 for symbol +Cmp
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 1.000000 for symbol +Der/minen
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 50000.000000 for symbol +Use/SpellNoSugg
/usr/local/bin/hfst-strings2fst -j < easteregg.default.desktop.txt \
> .generated/easteregg.default.desktop.hfst
/usr/bin/printf "\
set encode-weights ON \n\
read regex [ \
@\"filters/remove-usage-tags.hfst\" \
.o. @\".generated/generator-desktopspeller-gt-norm-tag_weighted.hfst\" \
.o. @\"filters/remove-word-boundary.hfst\" ] \
| @\".generated/easteregg.default.desktop.hfst\" \
; \n\
save stack .generated/generator-desktopspeller-gt-norm.tmp.hfst\n\
quit\n" | /usr/local/bin/hfst-xfst -p -v
Using default output format OpenFst with tropical weight class
Using OpenFst's tropical weights as output
Reading from standard input...
variable encode-weights = ON
hfst[0]: ? bytes. 131751 states, 334564 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-desktopspeller-gt-norm.tmp.hfst .generated/generator-desktopspeller-gt-norm.hfst
cp .generated/generator-desktopspeller-gt-norm.hfst generator-desktopspeller-gt-norm.hfst
/usr/local/bin/hfst-invert -v -i .generated/generator-desktopspeller-gt-norm.hfst \
| /usr/local/bin/hfst-prune-alphabet -v \
| /usr/local/bin/hfst-remove-epsilons -v -o .generated/analyser-desktopspeller-gt-norm.hfst
Reading from .generated/generator-desktopspeller-gt-norm.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to .generated/analyser-desktopspeller-gt-norm.hfst
Inverting .generated/generator-desktopspeller-gt-norm.hfst...
Pruning invert(UNNAMED)...
Removing epsilons prune-alphabet(invert(UNNAMED))...
cp .generated/analyser-desktopspeller-gt-norm.hfst analyser-desktopspeller-gt-norm.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/editdist.py \
--verbose \
\
--epsilon='@0@' \
--default-weight=10 \
--regex \
--input=editdist.default.txt \
--output-file=editdist.default.regex \
"/usr/local/bin/hfst-regexp2fst" --format=foma \
-v -S editdist.default.regex -o editdist.default.hfst
Reading from editdist.default.regex, writing to editdist.default.hfst
Compiling expression #1
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
[ then the previous line was repeated 4000 times] <== comment, TT
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '@_UNKNOWN_SYMBOL_@' is not an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
[ then the previous line was repeated 40 times ] <== comment, TT
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
Compiling expression #2
grep -v '^#' strings.default.txt | grep -v '^$' | cut -f1-2 \
| /usr/local/bin/hfst-strings2fst -j \
| /usr/local/bin/hfst-concatenate .generated/anystar.hfst - \
| /usr/local/bin/hfst-concatenate - .generated/anystar.hfst \
-o .generated/strings.txt.default.hfst
cp .generated/strings.txt.default.hfst strings.txt.default.hfst
cp -f strings.txt.default.hfst strings.all.default.hfst
/usr/local/bin/hfst-disjunct strings.all.default.hfst editdist.default.hfst \
| /usr/local/bin/hfst-repeat -f 1 -t 2 \
-o editdist.all.default.hfst
/usr/local/bin/hfst-disjunct: warning: Transducer type mismatch in strings.all.default.hfst and editdist.default.hfst; using former type as output
grep -v '^#' final_strings.default.txt | grep -v '^$' | cut -f1-2 \
| /usr/local/bin/hfst-strings2fst -j \
--format=openfst-tropical \
-o .generated/final_strings.txt.default.hfst
cp .generated/final_strings.txt.default.hfst final_strings.txt.default.hfst
cp -f final_strings.txt.default.hfst final_strings.all.default.hfst
printf "\
[ @\"words.default.hfst\" \
| \
[ \
\
@\"editdist.all.default.hfst\" \
( @\"final_strings.all.default.hfst\" ) \
] \
];" \
| /usr/local/bin/hfst-regexp2fst -S -E \
| /usr/local/bin/hfst-push-weights --push=initial \
| /usr/local/bin/hfst-fst2fst -f olw \
-o errmodel.default.hfst
/usr/local/bin/hfst-invert -v -i .generated/generator-mobilespeller-gt-norm.hfst -o .generated/analyser-mobilespeller-gt-norm.hfst
Reading from .generated/generator-mobilespeller-gt-norm.hfst, writing to .generated/analyser-mobilespeller-gt-norm.hfst
Inverting .generated/generator-mobilespeller-gt-norm.hfst...
cp .generated/analyser-mobilespeller-gt-norm.hfst analyser-mobilespeller-gt-norm.hfst
/usr/local/bin/hfst-compose -1 filters/remove-error-strings.hfst -2 generator-desktopspeller-gt-norm.hfst -F \
| /usr/local/bin/hfst-project \
-v --project=lower \
| /usr/local/bin/hfst-remove-epsilons -v \
| /usr/local/bin/hfst-disjunct -v - .generated/easteregg.default.desktop.hfst \
| /usr/local/bin/hfst-push-weights -v --push=initial \
| /usr/local/bin/hfst-fst2fst -v -f olw \
-o acceptor.default.hfst
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from - and .generated/easteregg.default.desktop.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to acceptor.default.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Projecting second compose(UNNAMED, UNNAMED)...
Removing epsilons project-2nd(compose(UNNAMED, UNNAMED))...
Disjuncting remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))) and strings(?)...
Pushing towards start union(remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))), strings(?))...
Converting push-weights-i(union(remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))), strings(?)))...
rm -f fkv.zhfst
/usr/local/bin/gmkdir -p build/fkv.zhfst
rm -f build/fkv.zhfst/*
cp index.xml build/fkv.zhfst/index.xml
cp acceptor.default.hfst build/fkv.zhfst/acceptor.default.hfst
cp errmodel.default.hfst build/fkv.zhfst/errmodel.default.hfst
cd build/fkv.zhfst/ && /usr/bin/zip -9 ../../fkv.zhfst *
adding: acceptor.default.hfst (deflated 81%)
adding: errmodel.default.hfst (deflated 80%)
adding: index.xml (deflated 55%)
/usr/local/bin/gmkdir -p 3
cp -f fkv.zhfst 3/
ln -f fkv.zhfst fkv-desktop.zhfst
rm editdist.all.default.mobile.hfst .generated/analyser-speller-gt-norm.hfst final_strings.txt.default.hfst strings.txt.default.hfst editdist.default.mobile.hfst strings.txt.default.mobile.hfst predict.default.regex .generated/strings.txt.default.hfst final_strings.all.default.mobile.hfst .generated/editdist.default.mobile.hfst editdist.default.mobile.regex .generated/analyser-fstspeller-gt-norm.hfst words.default.mobile.hfst final_strings.all.default.hfst strings.all.default.mobile.hfst predict.default.hfst easteregg.default.mobile.suggtxt .generated/final_strings.txt.default.hfst strings.all.default.hfst final_strings.txt.default.mobile.hfst .generated/generator-speller-gt-norm.hfst
Making all in hyphenators
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in grammarcheckers
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in tests
make[3]: Nothing to be done for `all'.
Making all in mt
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in apertium
Making all in filters
make[4]: Nothing to be done for `all'.
Making all in tagsets
make[4]: Nothing to be done for `all'.
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in cgbased
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in tts
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in test
make[3]: Nothing to be done for `all'.
make[2]: Nothing to be done for `all-am'.
Making all in docs
make[1]: Nothing to be done for `all'.
Making all in test
Making all in src
Making all in phonology
make[3]: Nothing to be done for `all'.
Making all in orthography
make[3]: Nothing to be done for `all'.
Making all in morphology
make[3]: Nothing to be done for `all'.
Making all in syntax
make[3]: Nothing to be done for `all'.
make[3]: Nothing to be done for `all-am'.
Making all in tools
Making all in spellcheckers
Making all in fstbased
Making all in desktop
Making all in hfst
make[6]: Nothing to be done for `all'.
make[6]: Nothing to be done for `all-am'.
Making all in mobile
make[6]: Nothing to be done for `all-am'.
make[5]: Nothing to be done for `all-am'.
make[4]: Nothing to be done for `all-am'.
Making all in mt
Making all in apertium
make[4]: Nothing to be done for `all'.
make[4]: Nothing to be done for `all-am'.
Making all in hyphenators
Making all in fstbased
make[4]: Nothing to be done for `all'.
Making all in patternbased
make[4]: Nothing to be done for `all'.
make[4]: Nothing to be done for `all-am'.
make[3]: Nothing to be done for `all-am'.
make[2]: Nothing to be done for `all-am'.
flammie commented
Compiling expression #1
warning: ' ' is an ordinary symbol in hfst
warning: ' ' is an ordinary symbol in hfst
It works on my linux and compilation is mostly the same but this doesn't appear...
flammie commented
newest giella-core should compile more with openfst and in hidden work dir