BJTUSensor/Phi-OTDR_dataset_and_codes

Is the calculation for Precision, NAR, and F1_score correct?

Opened this issue · 1 comments

The code in the "das_data_svm.py" file and "das_data_cnn.py" file for calculating metrics is currently written as follows:

“Acc = (C[0][0] + C[1][1] + C[2][2] + C[3][3] + C[4][4] + C[5][5]) / sum(C[0] + C[1] + C[2] + C[3] + C[4] + C[5])
print('acc: %.3f' % Acc)
lie_he = sum(C, 1) - 1
for i in range(1, 7):
Precision = C[i - 1][i - 1] / lie_he[i - 1]
NAR = (sum(C[i - 1]) - C[i - 1][i - 1]) / sum(C[i - 1])
F1_score = 2 * C[i - 1][i - 1] / (lie_he[i - 1] + sum(C[i - 1]))
print('precision_%d: %.3f' % (i, Precision))
print('NAR_%d: %.3f' % (i, NAR))
print('F1_score_%d: %.3f' % (i, F1_score))”

I believe this section should be modified as follows:
“rows_sum = sum(C, axis=1)
columns_sum = sum(C, axis=0)

for i in range(1, 7):
Precision = C[i - 1, i - 1] / columns_sum[i - 1]
NAR = (rows_sum[i - 1] - C[i - 1, i - 1]) / rows_sum[i - 1]
F1_score = 2 * C[i - 1, i - 1] / (rows_sum[i - 1] + columns_sum[i - 1])
print('Precision_%d: %.3f' % (i, Precision))
print('NAR_%d: %.3f' % (i, NAR))
print('F1_score_%d: %.3f' % (i, F1_score))

Am I doing the right thing? Does anyone else think there's something wrong with this part of the calculation?

I apologize for not responding to your question promptly. Your code implements the same function as our original code, but after subsequent research, we modified the evaluation metrics for Phi-OTDR event recognition, the new metrics include Accuracy, NAR(Nuisance Alarm Rate), FNR(False Negative Rate ), Precision, Recall and F1-Score, see README.md and "das_data_svm.py" , "das_data_cnn.py" for details.