Matlab code for algorithm performance comparison: Friedman test and adjusted p value by Hommel procedure
adjusted_p.m
-
import your data matrix, for example, cal.xlsx as the matrix cal, not table! And Note that Matlab always think the first row is the header!
-
row: test problems
-
column: algorithms, the first column is the control method
-
cell: test result of a algorithm (column) at a test problem (row)
-
You can refer to the Table III in the paper: A Federated Data-Driven Evolutionary Algorithm Link
-
-
run [ranks, original_p, ad_p] = adjusted_p(cal)
-
return: Friedman test, p value without adjusted, adjusted p value by Hommel procedure at 0.05 significance level, please note that the first column must the control method!
- save result vector of 20 independet runs, e.g. x=[run1, run2,..., run20]' (column vector), y=[b_run1, b_run2,...,run20]'
- matlab: [p, h]=ranksum(x, y, 0.05), if h=0, no significant difference, else if h=1, significant difference.