PubLayNet is a large dataset of document images, of which the layout is annotated with both bounding boxes and polygonal segmentations. The source of the documents is PubMed Central Open Access Subset (commercial use collection). The annotations are automatically generated by matching the PDF format and the XML format of the articles in the PubMed Central Open Access Subset. More details are available in our paper "PubLayNet: largest dataset ever for document layout analysis.".
25/Nov/2019 - PubTabNet is released! PubTabNet is a large dataset for image-based table recognition, containing 568k+ images of tabular data annotated with the corresponding HTML representation of the tables. Table regions are identified using the same algorithm that generates PubLayNet.
01/Nov/2019 - Our paper "PubLayNet: largest dataset ever for document layout analysis." receives the best paper award at ICDAR 2019!
31/Oct/2019 - PubLayNet migrates from Box to IBM Data Asset eXchange.
In parallel with data migration, the pre-trained Faster-RCNN and Mask-RCNN models are under legal assessment and will be released on IBM Model Asset eXchange (MAX).
The ground truth of test will not be release, as we want to keep it for a competition in the future. We will offer a service for people to submit and evaluate their results soon.
Images and annotations can be downloaded here. The training set is quite large, so two options are offerred. We split the training set into 7 batches, which can be separately downloaded. Or you can also download the full set at once.
If direct download in browser is unstable or you want to download the data from the command line, you can use aws s3 API, curl or wget to download the data.
aws --endpoint-url=https://dax-assets-dev.s3.us-south.cloud-object-storage.appdomain.cloud s3 cp s3://dax-publaynet/1.0.0/publaynet.tar.gz <YOUR_TARGET_DIR>/publaynet.tar.gz
curl -o <YOUR_TARGET_DIR>/publaynet.tar.gz https://dax-assets-dev.s3.us-south.cloud-object-storage.appdomain.cloud/dax-publaynet/1.0.0/publaynet.tar.gz
wget -O <YOUR_TARGET_DIR>/publaynet.tar.gz https://dax-assets-dev.s3.us-south.cloud-object-storage.appdomain.cloud/dax-publaynet/1.0.0/publaynet.tar.gz
The annotation files follows the json format of the Object Detection task of MS COCO
A Jupyter notebook is provided to generate the following visualization of the annotations of 20 sample pages.
