Name of Quantlet: Q-Kolleg2016
Description: This folder provides quantlets for Q-Kolleg 2016. Files:
-
This project consists of 2 approaches to text mining that each require multiple steps:
- Scraping of data from CRC 649 (01_Scraping)
- Converting scraped data into usable text format for R (02_PDF_to_txt)
- Combine text data in a corpus (03_Stem_TDM_TFIDF)
- Run the analyses (04_Analysis)
-
Every step depends on the outcome of the previous step, but starting points are provided for every step in form of .Rdata-files. The .Rdata-files contain the outcome of the previous step. This allows you to focus on one particular step of the process, without having to deal with preparation steps.
-
Two approaches are used to find coherent structures among the articles:
- Clustering of academic articles based on their abstracts
- Topic modeling (LDA) based on the entire article
-
Files used for approach 1 are labeled as "Abstracts_", whereas files used for approach are called "Articles_"
-
In the analysis of both approaches, the outcomes will be compared to JEL (Journal of Economic Literature) codes and project groups of the CRC 649.
Keywords: Topic modeling, R, Webscraping, Latent Dirichlet Allocation, Textmining, Clustering, Wordcloud
Author: Ken Schröder, Johannes Stoiber
Submitted: 2016/11/14