/scholarec

Recommendation engine for scholarly articles

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

scholarec

Recommendation of Scholarly Works

Build Status Dependency Status Zenodo DOI for github

This software has been built due to a need felt for a proper recommendation system for publicly available scholarly/research works.

It classifies documents and uses personalization features and content-based algorithm to suggest/recommend similar ones, possibly of interest to the user.

Note: Currently, full-functionality is offered by combining this package and another one, that offeres web interface (Django-based).

Inspired from an older project researchlei

Installation

  • Option 1: Install package only..
    $ pip install scholarec
  • Test through:
    $ python -m scholarec
  • Option 2: clone this repository..
    $ git clone https://github.com/arcolife/scholarec.git
    $ cd scholarec/
    $ sh setup.sh
  • See INSTALL for detailed instructions.

Test

  • To see if the scripts runs without error:
    $ ./tests/run-tests.sh
    $ ./tests/test.py
  • Optionally, to test if installed, look for a description on executing:
    $ python -m scholarec

Usage

  • To use the module in a Python script, simply import:
    import scholarec

  • To check a sample run output, open log/sample_run.txt

  • To go for a sample run:

    $ ./tests/query_parse

Note: For developing a small database from arXiv, you need to run the query_parse script and accept "Extract PDF" option for extracting related pdf's, converting them to plain text and extracting interesting words that would later be used for recommendations and suggestions.

  • A simple arXiv API call can be achieved by executing the following sample code:
import scholarec
from scholarec.base.arxiv import DocumentArXiv
url = "http://export.arxiv.org/api/query?search_query=all:%22higgs%22&start=0&max_results=2"
from urllib2 import urlopen
query_xml = urlopen(url)
doc = DocumentArXiv(query_xml)
data_dict = doc.extract_tags()
for entry_id in data_dict.keys():
    print "ID: %s" % (entry_id)
    print(data_dict[entry_id]), "\n"

FAQ

Q. What data interchange file formats have been used?

A. Data conversion from XML to JSON as well as in XML itself.

Q. What are the Data sources?

A. Dataset currently taken from arXiv. Future: DBLP/Google Scholar.

Q. How is the Data dealt with?

A. ElasticSearch/MongoDB for search and storage

LICENSE

GPL V3