/clustering_manager

Clustering task manager, queue control, and delivering results.

Primary LanguagePythonMIT LicenseMIT

Clustering-Manager

A basic GUI for local clustering task management. Using the clustering optimization platform for easily parameter selection, progress tracking, and delivering results. The user-interface generate the demand queue and the manager read the tasks and control the data clustering by demand.

Requirements

Install

Configure environment variables, install the clustering algorithm, generate the basic server, and set the linux service:

APP.CONFIG

Initial parameters header configurations.

  • Create the input path and set the clustering input location.

    Receives the user data and parameters:

    File app.py : app.config['UPLOAD_LOCATION'] = "/var/www/html/clustering/input/"

    File control_clstropt.py : LOCATION_QUEUE = "/var/www/html/clustering/"

  • Set queue location:

    File app.py : app.config['CTRL_LOCATION'] = "/var/www/html/clustering/"

    File control_clstropt.py : LOCATION_QUEUE = "/var/www/html/clustering/""

  • Create the input path and set the clustering output location.

    Return the cluster information, including the cluster ids and report:

    File app.py : app.config['DOWNLOAD_LOCATION'] = "/var/www/html/clustering/output/"

    File control_clstropt.py : "/var/www/html/clustering/output/"

    Set IP access on output public folder.

    File app.py : app.config['WEB_ENV'] = "http://"+HOST_IP+"/clustering/output/"

  • Python prefix called for clustering:

    File control_clstropt.py : PYTHON_MODE = "python3"

  • Process identifier for Cron check:

    File control_clstropt.py : PIDFILE = "/tmp/control_clstropt.pid"

  • The clustering optimization absolute path:

    File control_clstropt.py : CLSTR_APP = "/home/USER/clustering/main_clustering.py"

Clustering

Install the clustering optimization at the control_clstropt.py absolute path. Insert the file control_clstropt.py inside these path.

Server

Install Flask and Waitress. Configure app.py settings on serve() function at app.py.

Generate the linux service

Create /etc/init/clstropt and replace /path/to/ for app.py location:

# clstropt.conf
start on filesystem
exec python3 /path/to/app.py

Create /etc/init.d/clstropt file

# clstropt.conf
start on filesystem
exec python3 /path/to/app.py

Manage the GUI service activity using:

sudo service clstropt start

sudo service clstropt stop

Cron

Set time crontab cycles to execute the manager. The process remove overlap in a built-in function using ps command.

10 * * * * python3 /path/to/control_clstropt.py > /path/to/error.log