Method and metric

[Michael-E-Rose]

I sense potential in this package and I am inclined to use it in the future. Keep on the good work! Consider publishing the software in the SoftwareX journal.

I have a number of questions that I could not answer from the arguably short documentation:

• It seems to me that I can either pick DBSCAN or say silhouette score, but not both at the same time. This seems odd to me because DBSCAN is a method whose results could be used with the silhouette score.
• Related to that question: How are clusters evaluated if I pick DBSCAN or HDBSCAN, and how are clusters computed if I pick silhouette score or the Davies-Boulin index.
• How could I choose different distance metrics to plug into e.g. DBSCAN or HDBSCAN?
• How do I see which parameters got chosen?

[erdogant]

Dear Michael, Thank you for the tip and noticing these good points! I was readily working on a more extensive documentation page. This will take some time to finalize. But for now: 1. Thats a fair point. I need to restructure and put (h)dbscan as a separate input parameter. At the moment dbscan uses silhouette score for the evaluation. Other evaluation approaches for dbscan are not possible at the moment. 2. dbscan uses -as a standard- silhouette scores. When you choose “method” as silhouette, dbindex or derivative, you can set the metric and linkage (all scipy namings should be possible). The distance metric to use. The distance function can be ‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘cityblock’, ‘correlation’, ‘cosine’, ‘dice’, ‘euclidean’, ‘hamming’, ‘jaccard’, ‘jensenshannon’, ‘kulsinski’, ‘mahalanobis’, ‘matching’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’. 3. With metric! 4. You can plot the results. Or do you want to see something else? Verstuurd vanaf mijn iPhone

…

Op 23 aug. 2020 om 13:40 heeft Michael E. Rose ***@***.***> het volgende geschreven:  I sense potential in this package and I am inclined to use it in the future. Keep on the good work! Consider publishing the software in the SoftwareX journal. I have a number of questions that I could not answer from the arguably short documentation: It seems to me that I can either pick DBSCAN or say silhouette score, but not both at the same time. This seems odd to me because DBSCAN is a method whose results could be used with the silhouette score. Related to that question: How are clusters evaluated if I pick DBSCAN or HDBSCAN, and how are clusters computed if I pick silhouette score or the Davies-Boulin index. How could I choose different distance metrics to plug into e.g. DBSCAN or HDBSCAN? How do I see which parameters got chosen? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Michael-E-Rose]

Thanks for the answers! Questions 1 and 3 are about very similar things, so I take from it that my current application (DBSCAN on geo-coordinates, which require haversine distance) doesn't work with clusteval.

For the fourth question I wanted to know which values of hyperparameters got chosen in the end: With e.g. DBSCAN I assume multiple values for "eps" are tested, so I am keen learning which one yielded the best result.

[erdogant]

For dbscan the model iterates over various epsilon scores and computes the silhouette score for each. At the end, the index of the optimal silhouette score is used to determine the number of clusters. With that you can find back the specific epsilon score. See this figure. Left y-axis are the silhouette scores, right y-axis the cluster numbers and x-axis the epsilon. Maybe a scatterplot would be usefull too. https://github.com/erdogant/clusteval/blob/master/docs/figs/fig5_dbscan.png?raw=true On the other hand, there are also specific methods developed for dbscan optimization, which are also on my list to integrate in this library. See this github: https://github.com/christopherjenness/DBCV Verstuurd vanaf mijn iPhone

…

> Op 24 aug. 2020 om 14:56 heeft Michael E. Rose ***@***.***> het volgende geschreven:  Thanks for the answers! Questions 1 and 3 are about very similar things, so I take from it that my current application (DBSCAN on geo-coordinates, which require haversine distance) doesn't work with clusteval. For the fourth question I wanted to know which values of hyperparameters got chosen in the end: With e.g. DBSCAN I assume multiple values for "eps" are tested, so I am keen learning which one yielded the best result. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[erdogant]

I released a new update which has the separate input parameter for the clustering method and evaluation methods!
More information can be found here: https://github.com/erdogant/clusteval/releases/tag/2.0.0

Update to the latest release with:

pip install -U clusteval

The version should be >= 2.00

import clusteval
print(clusteval.__version__)