ckyeungac/deep-knowledge-tracing-plus

m1 and m2 calculation

MateuszOtmianowski opened this issue · 0 comments

I've got a quick question about the way in which you have calculated m1 and m2 metrics. You have written in your paper that those two metrics aim "to measure the consistency between the observed input and the change in the corresponding prediction". So if we are interested in how the input impacts the prediction, why when calculating coefficient, you use y (y_corr_batch and y_seq_batch, so target values) instead of corresponding x values?

I have in mind this part of your code:

coefficient = np.sum( (np.power(base, 1 - y_corr_batch[:, 1:, :])) * y_seq_batch[:, 1:, :], axis=2)

m1 = np.sum(
    coefficient * np.sign(np.sum(
        (pred_seqs[:, 1:, :] - pred_seqs[:, :-1, :]) * y_seq_batch[:, 1:, :], #y_t-y_{t-1}
        axis=2
        ))
    )
m2 = np.sum(
    coefficient * np.sum(
        (pred_seqs[:, 1:, :] - pred_seqs[:, :-1, :]) * y_seq_batch[:, 1:, :],
        axis=2
        )
    )