GENIA Tagger
Tsujii laboratory
University of Tokyo
1. How to build
You need gcc to build the tagger.
% make
if you encounter errors with hash, try commenting out "#define USE_HASH_MAP"
in "maxent.h".
2. How to use
Prepare a text file containing one sentence per line, then
% ./geniatagger < TEXTFILE > TAGGEDTEXT
The tagger outputs the base forms, part-of-speech (POS) tags, chunk tags,
and named entity (NE) tags in the following tab-separated format.
word1 base1 POStag1 chunktag1 NEtag1
word2 base2 POStag2 chunktag2 NEtag2
: : : :
Chunks and named entities are represented in the IOB2 format (B for BEGIN,
I for INSIDE, and O for OUTSIDE). The named entity tagger is trained on
the NLPBA data set [3], which contains five semantic classes (DNA, RNA,
cell_line, cell_type, and PROTEIN).
2.1 Tokenization
This tagger tokenizes the sentence with almost the same policy as the
upenn tokenizer (http://www.cis.upenn.edu/~treebank/tokenizer.sed).
If you don't want the tagger to perform tokenization, use -nt option.
In that case, the input sentences should be already tokenized by your
own tokenizer with white spaces.
References
[1] Yoshimasa Tsuruoka and Jun'ichi Tsujii, Bidirectional Inference with
the Easiest-First Strategy for Tagging Sequence Data, Proceedings of
HLT/EMNLP 2005, pp. 467-474.
[2] Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John
McNaught, Sophia Ananiadou, and Jun'ichi Tsujii, Developing a Robust
Part-of-Speech Tagger for Biomedical Text, Advances in Informatics -
10th Panhellenic Conference on Informatics, LNCS 3746, pp. 382-392, 2005
[3] http://research.nii.ac.jp/~collier/workshops/JNLPBA04st.htm
Please send questions and comments to tsuruoka@is.s.u-tokyo.ac.jp.
---------------------------------
Tsujii laboratory,
Department of Computer Science,
University of Tokyo
http://www-tsujii.is.s.u-tokyo.ac.jp/
ecnumjc/geniatagger
- part-of-speech tagging, shallow parsing, and named entity recognition for biomedical text -
GroffNOASSERTION