philihp/openskill.js

Ranking producing strange results

logan-anderson opened this issue · 2 comments

Example

import {rate, rating as openRating, ordinal } from 'openskill'

// 12 teams of 1
const teams = [[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()]]


// Lowest rank first
const rank = [1,2,2,4,5,5,7,8,9,9,11,12]

const all = rate(teams,{rank})

all.forEach(x=>console.log(ordinal(x[0])))

This outputs

1.9887943085056428
0.7351607181385624
0.7351607181385624
1.5932055407029928
0.2720169616753587
0.2720169616753587
1.0192331720170102
0.6235031893289182
-0.9496298411755326
-0.9496298411755326
-0.8953795301204437
-3.0470369446801193

Which is odd because the two people in 2-3 tied, received a rating of 0.7351607181385624, but the person who ranked 4th got a higher rating of 1.5932055407029928.

If I do the inverse (with score) I get a similar result.

const score = [12,11,11,9,8,8,6,4,3,2,1]
const open2 = rate(teams,{score})
open2.forEach(x=>console.log(ordinal(x[0])))

outputs

1.9887943085056428
0.7351607181385624
0.7351607181385624
1.5932055407029928
0.2720169616753587
0.2720169616753587
1.0192331720170102
0.6235031893289182
0.12619886610430697
-0.542803641476798
-1.5642761925161288
-3.7159336070758044

If I use thurstoneMostellerFull I get a more expected result.

const teams = [[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()],[openRating()]]
const rank = [1,2,2,4,5,5,7,8,9,9,11,12]
const open3 = rate(teams,{rank, model: thurstoneMostellerFull})
open3.forEach(x=>console.log(ordinal(x[0])))

outputs

71.00770829746648
58.39200967088472
58.39200967088472
45.776231044302946
33.16053241772118
33.16053241772118
20.54475379113941
12.13426137341823
-0.48143725316354136
-0.48143725316354136
-13.097215879745306
-21.50770829746648

Is there something I am missing? Could this be an issue with the Plackett-Luce model?

Thanks for your time!

This seems to be a duplicate of #332. I've explained how ordinal works with a diagram there to get better intuition.

Hope that answered things, if not please feel free to reopen... i'll get around to documenting sometime