/f8a-hpf-insights

HPF Matrix Factorization for Maven companion recommendation

Primary LanguageJupyter NotebookGNU General Public License v3.0GPL-3.0

f8a-hpf-insights

(fabric8-analytics-hpf-insights)

HPF Matrix Factorizations for companion recommendation. HPF- Hierarchical Poisson Factorization

Index:

Supported ecosystems:

  • Maven - Last trained at: 2018-08-08 11:30 IST(UTC +5:30)

Build upon:

To run locally via docker-compose:

  • Setup Minio and start Minio server so that hpf-insights is loaded as a folder inside it upon running. To use AWS S3 instead of Minio add your AWS S3 credentials in the next step instead of Miino credentials.
  • Create a .env file and add credentials to it.
  • In the .env set the AWS_S3_ENDPOINT_URL to <blank> for using AWS S3 and to http://ip:port for using Minio.
  • source .env
  • docker-compose build
  • docker-compose up
  • curl http://0.0.0.0:6006/ should return status: ok

To run on dev-cluster:

  • cp secret.yaml.template secret.yaml
  • Add your AWS S3 credentials to secret.yaml
  • oc login
  • oc new-project hpf-insights
  • oc create -f secret.yaml
  • oc process -f openshift/template.yaml -o yaml|oc create -f - If you want to update the template.yaml and redeploy it, then do oc process -f openshift/template.yaml -o yaml|oc apply -f - Use apply instead of create for subsequent re-deployments.
  • Go your Openshift console and expose the route
  • curl <route_URL> should return status:ok

Unit Tests

There's a script named runtests.sh that can be used to run all unit tests. The unit test coverage is reported as well by this script.

Usage:

./runtests.sh

To run load testing for recommendation API:

  • pip install locustio==0.8.1
  • Bring up the service.
  • locust -f perf_tests/locust_tests.py --host=<URL of the service>

Footnotes:

Coding standards:

  • You can use scripts run-linter.sh and check-docstyle.sh to check if the code follows PEP 8 and PEP 257 coding standards. These scripts can be run w/o any arguments:
./run-linter.sh
./check-docstyle.sh

The first script checks the indentation, line lengths, variable names, whitespace around operators etc. The second script checks all documentation strings - its presense and format. Please fix any warnings and errors reported by these scripts.

Code complexity measurement:

The scripts measure-cyclomatic-complexity.sh and measure-maintainability-index.sh are used to measure code complexity. These scripts can be run w/o any arguments:

./measure-cyclomatic-complexity.sh
./measure-maintainability-index.sh

The first script measures cyclomatic complexity of all Python sources found in the repository. Please see this table for further explanation how to comprehend the results.

The second script measures maintainability index of all Python sources found in the repository. Please see the following link with explanation of this measurement.

You can specify command line option --fail-on-error if you need to check and use the exit code in your workflow. In this case the script returns 0 when no failures has been found and non zero value instead.

Dead code detection

The script detect-dead-code.sh can be used to detect dead code in the repository. This script can be run w/o any arguments:

./detect-dead-code.sh

Please note that due to Python's dynamic nature, static code analyzers are likely to miss some dead code. Also, code that is only called implicitly may be reported as unused.

Because of this potential problems, only code detected with more than 90% of confidence is reported.

Common issues detection

The script detect-common-errors.sh can be used to detect common errors in the repository. This script can be run w/o any arguments:

./detect-common-errors.sh

Please note that only semantical problems are reported.

Check for scripts written in BASH

The script named check-bashscripts.sh can be used to check all BASH scripts (in fact: all files with the .sh extension) for various possible issues, incompatibilies, and caveats. This script can be run w/o any arguments:

./check-bashscripts.sh

Please see the following link for further explanation, how the ShellCheck works and which issues can be detected.

Additional links: