ML4BIHECOL

A project funded by: FEDER/Ministerio de Ciencia, Innovación y Universidades – Agencia Estatal de Investigación/ _Proyecto TEC2017-83838-R.

Machine learning tools for the analysis of large document collections. A brief description of each of them is given, even though each has it's own documentation.

The available repositories, divided by tasks are:

Spark implementation of parallel algorithms:

DistributedCWLM: distributed, interpretable regression algorithm in Spark.
SVM_spark: distributed, non-linear, semiparametric SVM implementation in Spark.
pysparkMVA: distributed implementation of Deep Learning for Margin Valuation Adjustment with regularization.

Topic Models tools:

TMtweets: topic modeling pipeline. Designed to work with tweets, but can be extended to other documents.
labelFactory: a generic application for labelling a subset of sites in a web site collection, or a subset of docs in a text collection.
one_def_classification: dataless text classification using definitions as labeled data.
WeakLabelModel: a library for training multiclass classifiers with weak labels.

Integration of modules:

PTL_data: ETL & lemmatization tool for PTL.
PdbManager: Managing mySQL databases (replaces older DB_expl).
dbManager: Python class for managing mySQL or sqlite databases.
menuNavigator: a generic template application to generate command-line menus.

Feature Selection and Feature Extraction:

regMVA: Deep Learning for Margin Valuation Adjustment with regularization.
SSHIBA: generalized Bayesian approach to feature extraction for heterogeneous data.
KSSHIBA: kernelized observation SSHIBA.

Graph synthesis, processing and analysis.

supergraph: a software package for the synthesis, analysis and processing of large graphs.

Instructions:

To clone the project with all their submodules:

git clone --recurse-submodules https://github.com/ML4DS/ML4BIHECOL

If you have already done

git clone https://github.com/ML4DS/ML4BIHECOL

you can do

git submodule update --init --recursive

Alternatively, you can use specific submodules only. The best way to work with a specific submodule is to clone the specific github project associated to the submodule

git clone https://github.com/URL/of/the/project