Convert HTML/XML Chemistry/Material Science articles into plain text.
See requirements.txt
.
Packages with versions specified in requirements.txt
are used to test the code.
Other versions are not fully tested but may also work.
- RSC (HTML)
- Springer (HTML)
- Nature (HTML)
- Wiley (HTML)
- AIP (HTML)
- ACS (HTML & XML)
- Elsevier (HTML & XML)
- AAAS (Science) (HTML)
Table parsing is supported but not for all publishers. For figures, only figure captions are parsed in the current version.
Fork this repo and clone it to your local machine;
To parse HTML files, run the following code:
python tests/parse_articles.py --input_dir </path/to/html/files> --parse_html
or
cd tests
python parse_articles.py config.json
where parameters are stored in file config.json
.
Add --parse_xml
to the argument list to enable xml parsing.
Due to the variety of HTML/XML documents, not all document can be successfully parsed. It would be helpful for our improvement if you can report the failed cases in the Issue section.
- HTML highlighting sometimes may fail when multiple entities start at the same position due to incorrect text span alignment.
- May fail to extract sections from Elsevier when section ids are
s[\d]+
instead ofsec[\d]+
, as mentioned in this issue. - Fails to extract abstracts from RSC due to updated HTML format, as mentioned in this issue.